P1810261 - Copy.jpg
lightfarm_F.png
IMG_0421v2.jpg
LightFarm Overview | Interactive Projection Mapping

LightFarm Overview | Interactive Projection Mapping

Play Video

About

LightFarm is a physical interactive experience that combines decision-making & problem solving to teach sustainable farming practices. Explore the life of a farmer through a physically constructed diorama powered by projection mapping and computer vision technology. The game combines exploration & problem solving to teach you about farming.

LightFarm was made in partnership with Marcatus QED, who provided our design challenge, feedback on the project and a source of real-world research.

RookiesWreah
intelwreath
unitywreath

Contributions and Roles

Game, UX and interaction design.

Programming, implementation, prototyping.

Team

5 members

2 research consultants from QED

Timeline

6 Months

(Sept 2019 - April 2020)

Tools

Software:

  • Unity C#

  • Adobe Suite

  • Webcam

  • Pico Projector

Our challenge

For our final thesis project, my team and I decided to partner with Marcatus QED, which provided our design challenge:

How might we create an interactive experience to teach farmers sustainable farming practices?

Background

With the help of MarcatusQED, we had access to specific research that they conducted and we were able to focus on a specific audience that the design challenge was aimed towards.

"500 million smallholder farming families produce 70% of the world’s food."

IMG_3014-process-s1200x800.jfif
Indian-Villages-Harvesting.jpg

These smallholder farming families can face a lot of challenges and obstacles in taking care of their life's work.

  • Limited access to formal education.

  • Limited access and unfamiliarity to technology, electricity and the internet.

  • Language barriers.

  • Harsh environmental factors (eg. weather) and life situations.

  • Family income is directly tied with produce yield from the farm.

MarcatusQED had made multiple efforts to create video teaching guides using projectors, tablets and other similar projects in efforts to teach smallholder farmers in India about sustainable practices.

unnamed.png

These instructional efforts were not as successful as intended since they ran into a number of issues when trying to help farmers switch to more sustainable farming practices.

  • Switching to sustainable practices requires change, and some farmers can be traditional in decision making

  • Switching requires more effort and doesn't always show improvement in yields until a few seasons later. This lack of immediate feedback can make it feel like it's not a good use of their hard work and time.

  • Not being aware of how their current practices affect the environment and diminishing long-term yields of their crops makes it an issue that doesn't need to be addressed right away.

  • Farmers aren't too engaged just by watching videos, it can feel too detached from their farm work and had a hard time retaining the information presented

workshop.png

Why LightFarm?

Follow a farmer's daily struggles and take care of your farm

By combining exploration and decision-making on an interactive farm, we created a memorable and engaging teaching tool. 

Animation4.gif
Animation5.gif

Easy to interact and play with

Our custom-designed controller makes physical interactions feel natural and intuitive.

Easy to learn about sustainable farming

Projection mapping allows us to visualize and convey a large amount of complex information about farming practices.

Animation3.gif

Process

process.png

Exploration

We brainstormed and tried a lot of different possible solutions of approaching this challenge to make an informed decision on which direction is best.
 

IMG_20191115_173530.jpg
unnamed2.png
unnamed2.png

To help us narrow down on which ideas to pursue, we decided to prototype a wide range of ideas which included AR, MR, VR and mobile implementations.

unnamed.gif
unnamed.jpg
unnamed (2).gif
70164580_470787350184190_611049969915055
sciencefair.gif

Creating a vision

After testing a number of prototypes, evaluating them, and considering a lot of tech constraints, we decided on a physical diorama for a number of reasons:
 

  • Interesting level design creates an explorative and narrative elements.

  • 3D diorama with a cross section helps communicate complex ideas about the soil.

  • Physical touch can help make interactions feel better.

unnamed (8).png
unnamed (1).png
IMG_20200122_134638.jpg

Projection Mapping

Since MarcatusQED had a number of projectors available for our use from their other teaching efforts, this gave us a great opportunity to try to use projection mapping as a way to display information on our diorama.

unnamed (1).png

By keeping the diorama generic and ambiguous, we would be able to project different environments and scenarios without having to switch the diorama.

howitworks.png

After a few tests with the tech, it become clear that projection mapping fit our project and we decided to pursue it.

IMG_20191101_102904.jpg
IMG_20191101_130502.jpg
plop.gif

Projection Mapping Challenges

As we started testing more complex shapes and builds for our diorama prototypes, some projection mapping challenges became clear:

unnamed (1).jpg
  • Projector has to be facing the diorama from the same general direction as observers to get the best projection experience. Projecting from a diagonal angle can cause the picture to stretch and lose the point of view effect. A solution to this is to stretch it inversely in-game.

IMG_20200308_183025.jpg
vlcsnap-2020-04-17-01h49m41s431.jpg
  • People's bodies and hands often block the projector if they walk and reach around the diorama outside of a specific intended area, creating lots of shadows.

IMG_20191205_134204.jpg
  • Projector has to be within 2 meters distance for reasonably high resolution on the projection.

 

IMG_20200220_171536.jpg

Interactions

To take full advantage of a physical diorama and create really solid interactions, we tested a wide range of input solutions.

unnamed (3).jpg
unnamed (4).png
unnamed (4).jpg
unnamed (2).jpg
unnamed (6).png
unnamed (11)_edited.png
unnamed (9).png
unnamed (5).png

Leapmotion seemed to give the best results based on initial testing and we started iterating with this interaction model.

ezgif.com-video-to-gif (3).gif

First prototypes

For our first prototype, we created some simple interactions to be able to test our input system.

unnamed (2).png
IMG_20191204_190416.jpg
MVIMG_20191205_123942.jpg

After our first testing sessions, the feedback became clear that our input system was not consistent enough, missing roughly one third of the touch inputs on the surface. We had to search for a new way to interact with the diorama in a more consistent manner.

unnamed (1).gif

Input rework

Using image recognition software, we found a way to create a touch input on a thin surface using the shadow of your fingertips. This worked great for our solution since the diorama would always be lit up by the projector and the shadows would be easy to detect.

image1.gif
IMG_20200130_174704.jpg

By making a hollow stand and using a combination of clear plastic and paper covers, the webcam at the bottom was able to track from multiple heights and surfaces of the diorama at once.

IMG_20200304_202258.jpg
IMG_20200304_202324.jpg
IMG_20200306_154922.jpg
IMG_20200225_213258.jpg
IMG_20200225_233031.jpg

Testing the new interactions and diorama showed a huge input consistency improvement and players focusing more on the experience.

IMG_20200226_112753.jpg
Animation2.gif

Product vs Game

It was a challenge to figure out how much content should be gamified and how much should be the content we are trying to teach.

We decided on a linear playthrough with progression based on a series of decisions on how to take care of your farm. These decisions would be focused on a specific lesson and would show you the different pros and cons of sustainable farming practices and how they will affect the farm over longer periods of time.

Game mechanics were designed around this structure to best highlight each decision and outcome.

unnamed (2).png

Outcomes

Takeaways
 

  • Lots of quick experimentation is important early on to make informed decisions.

  • Defining the boundaries between product and game design and when to use them.

  • Importance of space, lighting and view perspective on physical and projection mapping projects.

RookiesWreah
intelwreath
unitywreath

Projects

hands.png
illusionary photos.png
flexor.png