<aside>
Resources
https://docs.ml5js.org/#/reference/bodypose
https://thecodingtrain.com/tracks/ml5js-beginners-guide
</aside>
Based on the resources above, and the two detection models mentioned, I came up with 2 codes for ml5 pose detection, one for 2D(MoveNet model) and one for 3D(BlazePose model).
MoveNet model with 17 tracking key points.
BlazePose model with 33 tracking key points.
2D
https://editor.p5js.org/RayRayRule/sketches/2fXqDSHr3
3D
https://editor.p5js.org/RayRayRule/sketches/AmsdvLUf-
Since both models assign body parts with specific index, different pose condition can be set up as event trigger, utilizing the relative position of key points. Compared to MoveNet, BlazePose has more key points, enabling delicate movement tracking. However, it’s also slower for the same reason. In addition, BlazePose support 3D movement tracking, which is handy in project simulation.
Based on MoveNet, I made a “Rain Praying Program” 2D simulation. When the users’ hands are raised above shoulder, a raining event will be triggered.
https://editor.p5js.org/RayRayRule/sketches/RAlg7AKsi
Based on the 2D simulation, I built another 3D simulation for more “space information”. This is accomplished by the BlazePose model and the WebGL function.
https://editor.p5js.org/RayRayRule/sketches/SIodlL-jf
Also utilizing the capability of space distribution to build another 3D version for this project, just to see if the visual effect will be better if the rain is multiple layer.
https://editor.p5js.org/RayRayRule/sketches/k42DsiQne
<aside>
Possible adjustment
The rain visual is more interesting when two rain events are overlapping. The rain distribution for a group of rain can be compressed more, so that audience can see clearer pattern triggered from their action.
</aside>
p5js is an useful tool for building project simulation, and make design adjustment quickly. Especially with WebGL’s help, a 3D environment, including space and lighting, can be presented in an efficient way.
Next top, utilizing more WebGL function to create simulation with more real life detail: https://thecodingtrain.com/tracks/webgl