Palette Mesh Optimization

“Cof­fee Game” 1 is the work­ing ti­tle of our work in progress We­b­VR cof­fee house man­age­ment game. You will be able to make cof­fee, serve your cus­tomers and make mon­ey to buy new ma­chines. With the size of this game be­ing above the av­er­age We­b­VR game out there, we were re­quired to do some cus­tom op­ti­miza­tion to en­sure it runs smooth­ly on low­er end de­vices.

Dur­ing the writ­ing of this blog post, the game’s name was fi­nal­ized to “Barista Ex­press”!

The Café Scene

The scene the game plays in con­tains a lot of fur­ni­ture and small dec­o­ra­tive ob­jects to im­merse the us­er in his en­vi­ron­ment and the game’s set­ting.

As our team has lim­it­ed mod­el­ing ca­pac­i­ty, we de­cid­ed on a sim­ple low-poly and most­ly tex­ture­less art style that even a pro­gram­mer would be able im­i­tate when mak­ing as­sets. In this case, Flo­ri­an, the de­sign­er and de­vel­op­er of this game end­ed up al­so do­ing al­most all of the mod­el­ing work him­self! With the ex­cep­tion of a cou­ple of mod­els that we found on Google Poly or Sketch­fab, which is the sec­ond ben­e­fit of this very pop­u­lar art style.

Stats on the orig­i­nal glTF scene.
El­e­ment Amount
Ma­te­ri­als 43
Mesh­es 163
Nodes 167
Prim­i­tives 228
Tex­tures 1

Tex­ture­less means that ev­ery col­or is en­cod­ed in ma­te­ri­al pa­ram­e­ters, re­sult­ing in a lot of ma­te­ri­als. At the time of writ­ing the stat­ic parts of the scene con­tained what is list­ed in the ta­ble above.

Barista Express' café scene.

Draw call count spiked up to 200! This is not vi­able for most Card­board de­vices and did not run well on Ocu­lus GO or GearVR. And these are on­ly the stat­ic ob­jects! Thrash­ing the GPU with 200 tiny (be­cause of the low-poly style) draw calls is not the way you want to roll.

Batching Before Export

In­stead, how about merg­ing all the mesh­es to­geth­er in Blender be­fore ex­port? Since there are most­ly no tex­tures in the scene, the tex­tures do not need to be at­lassed and this could be very easy!

Join­ing all the mesh­es will great­ly re­duce the amount of mesh­es in the glTF file. There will still be one glTF prim­i­tive el­e­ment per ma­te­ri­al which will at best get the num­ber of draw calls down to 43. That is good, but we can do bet­ter.

Once you join the mesh­es in Blender, edit­ing is way hard­er. While an op­tion would be to cre­ate a script that joins all mesh­es in­to a joined mesh be­fore ex­port­ing to al­low keep­ing the un­joined mesh­es around, this is ad­di­tion­al has­sle you want to avoid for a good work­flow.

Vertex Colors

Since the on­ly dif­fer­ence be­tween our ma­te­ri­als is the base col­or, you will be think­ing this is a great fit for join­ing all mesh­es and con­vert­ing the ma­te­ri­als in­to ver­tex col­ors.

And you would be right with this as­sump­tion. The rea­sons we went for a slight­ly more com­pli­cat­ed so­lu­tion is a small im­prove­ment in file size, eas­i­er scene edit­ing work­flow and eas­i­er view­ing of the re­sult in gltf view­ers.

The glTF ex­porter for Blender ex­ports mesh col­or da­ta as three or four com­po­nent float vec­tors, i.e. 12 or 16 bytes of da­ta more per ver­tex. Since a lot of ver­tices will have the same col­ors, there is a lot of re­dun­dant in­for­ma­tion wast­ing space here. In­stead we are go­ing to use a PNG-like ap­proach by sav­ing all these col­ors in a tex­ture and us­ing a sim­ple in­dex in­to this col­or-pal­ette rather than the full col­or.

Ad­di­tion­al­ly, while ver­tex col­ors are sup­port­ed by glTF, not ev­ery tool, view­er or ren­der­ing frame­work triv­ial­ly sup­ports it.

Fi­nal­ly, hav­ing the op­ti­miza­tion as a sep­a­rate au­to­mat­ic stage af­ter ex­port (think of gulp watch­ing the ex­port di­rec­to­ry and au­to­mat­i­cal­ly pump­ing the da­ta through the op­ti­miza­tion pipe­line) rather than a man­u­al one be­fore is a nice im­prove­ment to de­vel­op­ment work­flow aswell.

Palette Mesh

Sup­pose you col­lect all the dif­fer­ent col­ors in your scene and cre­ate a tex­ture from those. The tex­ture will look some­thing like this:

An example palette texture, scaled up.

Sim­i­lar to an in­dex in­to a pal­ette for col­ormapped PNGs we can now “in­dex” in­to this tex­ture us­ing the tex­ture cor­rdi­nates. By set­ting UV co­or­di­nates so that each ver­tex re­ceives the col­or of ex­act­ly one of the pix­els we achieve the same look with a lot less da­ta. Since we have less than 64 dif­fer­ent UV co­or­di­nate val­ues, we can even get away with us­ing nor­mal­ized un­signed short da­ta for the tex­ture co­or­di­nates. (Nor­mal­ized un­signed byte da­ta for tex­ture co­or­di­nates is not sup­port­ed by glTF as the co­or­di­nates needs to be 4-byte aligned!)

We im­ple­ment­ed this op­ti­miza­tion us­ing gltf-pipe­line. “Gltf Pipe­line” is a node­js based com­mand line tool that op­ti­mizes glTF files, con­verts them to GLB and more.

Stats on the op­ti­mized scene.
El­e­ment Be­fore Af­ter
Ma­te­ri­als 43 7
Mesh­es 163 10
Nodes 167 10
Prim­i­tives 228 10
Tex­tures 1 2
File size 1673 kb 1747 kb

Af­ter run­ning the stat­ic scene through the pipe­line with our cus­tom stage, we end­ed up with the im­prove­ments you can see in the ta­ble above.

The re­main­ing draw calls and prim­i­tives are due to trans­par­ent/al­pha-blend­ed mesh­es not be­ing merged in­to the stat­ic mesh batch. This way we can keep draw­ing the big batch with an opaque ren­der­ing pipe­line and avoid is­sues with draw­ing-or­der for the trans­par­ent mesh parts.

The 74 kb in­crease in file size is owed to the added tex­ture co­or­di­nate da­ta and mesh­es that ap­pear mul­ti­ple times in the scene hav­ing their ge­om­e­try du­pli­cat­ed.

How To Use

Some of the changes we made to gltf-pipe­line will be con­trib­uted up­stream to the orig­i­nal repos­i­to­ry. But the pal­ette mesh op­ti­miza­tion does not fit, as it is very spe­cif­ic to the art style.

We open sourced the code for this cus­tom stage for gltf-pipe­line on Github vhit­er­ab­bit/gltf-pipe­line-stages. You can check­out the repos­i­to­ry and fol­low the ex­am­ple in the README. More cus­tom stages will even­tu­al­ly fol­low!

I hope you en­joyed the read and our code helps you build big­ger games in the fu­ture. Be sure to fol­low us on so­cial me­dia to catch no­tice of the re­lease of “Barista Ex­press” in the up­com­ing weeks—or sub­scribe to our news­let­ter.

Have fun build­ing VR brows­er games!

Jonathan of Con­struct Ar­cade.