Week 1: Initial idea development
Idea 1: Unreal Engine 5, interactive narrative, social anxiety
The main character will be a tiny particle escaping a noisy, overstimulating world in search of peace. As users drag the particle across different environments. There will be chaotic “noise zones” and “calm zones” like falling leaves or reflections on water. These interactions trigger subtle animations that reflect the emotional shift from anxiety to tranquility.
Idea 2: Blender, 2D style, pet love
The story will be told entirely through small objects and scenes. It traces the emotional arc of a cat-human relationship, without showing either. A ceramic bowl, a toy ball, a feather wand, a wool blanket, a book, and a collar bell represent a quiet moment: meeting in a cat café, playful companionship at home, silent comfort on hard days, and finally, loss.
Week 2: Vcam
UE’s VCam (Virtual Camera) allows users to simulate real-world camera operations inside Unreal Engine using a mobile device. It enables live camera movement, framing, and adjustment of settings like focus, aperture, and lens type, making it ideal for virtual production and cinematic previsualization.
Unreal Vcam (app)
Enable Plugins in UE: Virtual Camera, Live Link, and Apple ARKit
New Project idea: VJ project
Music choice: You’ve Got Control On Me
Music from #Uppbeat (free for Creators!):
https://uppbeat.io/t/neozoic/youve-got-control-on-me
License code: XP5KIWDHZCCIE0PF
This week I changed my project topic. Instead of continuing with the original idea, I decided to base my project on a piece of music. The plan is to find a track that inspires a clear visual direction, and then design the visual elements to match its mood and style. Later, I will use what we learn this semester about VJ performance to simulate a live VJ setup around this idea.
I found a music track that I think fits well. It has a haunting, emotional quality that immediately made me think of themes related to femininity and a sense of being restrained or suppressed. Along with that, I also found a dance performance reference that could help shape the animation rhythm and body movement in the visual scene.
This new direction gives me more space to work with symbolic and emotional imagery. It also connects well with the tools we’ll be exploring in class, like VJ setups. For the next step, I will start sketching visual ideas inspired by the music.
Dancing Clip Reference:
Week 3: NDisplay
NDisplay is a system in Unreal Engine used for rendering content across multiple screens or projection systems, often for immersive installations or virtual production. It allows users to synchronize rendering on several machines and displays, creating a seamless visual experience.
- Enable the nDisplay plugin in UE
- Create an nDisplay Config Asset to define screens (viewports), cameras
- Open the nDisplay Config Editor to adjust screen content and position
This week I continued working on the animation part of the project. Most of my time was spent refining the spline process to create smoother motion paths. It took several trials to get the timing and curves right, especially for camera movements and animated elements tied to the rhythm of the music.
Besides that, I also started collecting assets for the remaining scenes. These include environment elements like some chain models that fit my whole style and some plants models and also a spider rig. The goal is to build a consistent style across all parts of the VJ sequence while keeping the emotional tone I planned before.
For next week, I plan to start testing these assets inside UE and begin rough blocking of the remaining scenes.
Week 4: Project Feedback
This week, I completed the animation part and exported the sequence as a video. In After Effects, I added motion blur to enhance the movement’s smoothness. The result looks more polished.
I also began working on the second visual part. This week’s focus was on scene building and lighting tests. For the main subject of the scene, I reused a model I created during Term 1. I changed its material to better match the tone of the muisic. The model has distinct feminine features, which I feel align well with the emotional and thematic direction of this part of the soundtrack.
However, I’m not fully satisfied with the current lighting setup. The results feel a bit flat and don’t convey the mood I’m aiming for. Since this part will eventually blend into a stage-like environment, the lighting needs to feel deeper and more theatrical, while still clearly highlighting the model in the middle. I will continue to experiment with different lighting next week!
Week 5: Mocap
This week I had the chance to experience the motion capture. I wore the mocap suit and helped place markers on key joints, such as shoulders, elbows, knees, and ankles. These markers are essential for the system to accurately track body movement in 3D space.
After the setup, I performed a series of basic actions，like turning, lifting arms, to make sure each marker was being captured correctly. The goal was to get a clean data set that could later be used for character animation. Once the motion was recorded, we created the character model and applied the captured motion to it.
Seeing the animated model move based on real performance felt very intuitive and rewarding. It gave me a better understanding of how motion data connects with animation, and I will explore how this could be used in my own project later on (maybe FMP), especially for adding more expressive body movements.
This week I adjusted the lighting in my scene and added a Post Process Volume with bloom effect, which made a much better visual impact. The bloom helped emphasize the highlights and added more atmosphere to the darker areas. I also animated the lights to sync with the beats at the beginning of the music, creating a dramatic introduction. The animation simulates a flashlight effect, flickering and revealing parts of the model in sync with the audio.
I also started building a new scene this week, which will focus on a chain animation as the central visual element. The goal is to express a strong sense of constraint and restriction, continuing the mood style. To define the atmosphere, I tested several spotlights in different colors. I wanted to add a futuristic feeling to the scene, so I experimented with cooler tones like blue and purple, as well as higher contrast lighting setups.
Next week, I will continue adjusting the lighting in this scene to better match the mood I’m aiming for.
Week 6: Madmapper
MadMapper is used for mapping video into the space, controlling lights and LED bars.
This week I focused on refining the chain scene, both lighting and animation. I created a simple animation for the chains, with the main visual focus being a heart-shaped chain suspended in the center of the scene. This shape is meant to represent a sense of emotional confinement, in line with the project’s theme of suppressed or restrained feelings. I animated the heart chain to move slightly with the rhythm of the music’s drumbeat, simulating the heartbeat.
In terms of lighting, I made several adjustments to create a stronger mood. I added a key light to emphasize the heart-shaped chain as the focal point, a background light to define the environment, and some side lighting to give the scene more depth and contrast. One of the more experimental parts this week was trying out the Light Function. I created a custom material for a pink light source that pulses gently like a breathing effect. This helped bring a soft, surreal quality to the scene, enhancing the emotional tone.
Week 7: Mocap in UE
This week I started building my fourth scene, which focuses on Niagara particle animation. The particles in this setup are designed to change both shape and color over time, creating a shifting, organic visual that reacts to the rhythm and tone of the music. The goal is to convey a sense of transformation and emotional flow.
To emphasize the glowing particles as the main visual focus, I did not add any additional light sources in the scene. Since the particles are set to be self-emissive, they generate their own light, which creates a clean and minimalistic visual style.
Week 8: Tutorial
This week, I focused on adjusting the layout of my fourth scene. The new design takes inspiration from a futuristic, cyber-style flower field. Visually, it combines natural shapes with mechanical, glowing elements to create a surreal and symbolic environment. The constant morphing of the flowers reflects the core theme of restraint and resistance — a cycle of tension and release.
I also added camera animation to this scene. The movement begins wide and gradually pushes in toward a single flower, which acts as the focal point of this section. This flower serves as a visual anchor, much like the central figures in my other scenes. By doing this, I kept a consistent structure across the different segments of the project, where each has a central visual subject that connects back to the emotional flow of the music.
Next, I’ll start editing all my finished rendered videos!
Week 9: Resolume
This week I imported my four video sequences into Resolume Arena 6 and experimented with basic VJing. It was my first time trying live visual mixing, and I found that the different blending modes and dynamic layering effects provided a lot of interesting visual inspiration. Playing with opacity, and timing in real time made me think about new ways to connect the scenes and transition between moods.
However, I realized that my sense of rhythm is not strong enough to fully rely on live performance for this project. To maintain a smoother and more coherent visual flow, I decided to simulate the VJ effects in post-production instead. This gave me more control over timing and allowed me to fine-tune the transitions to match the mood of the music more precisely.
Even though I didn’t keep the live performance results, the experience helped me understand how real-time visuals can influence composition and pacing. I plan to integrate some of these layering ideas into my final edit.