LightFarm is a physical interactive experience that combines decision-making & problem solving to teach sustainable farming practices. Explore the life of a farmer through a physically constructed diorama powered by projection mapping and computer vision technology. The game combines exploration & problem solving to teach you about farming.

​

LightFarm was made in partnership with Marcatus QED, who provided our design challenge, feedback on the project and a source of real-world research.

Contributions and Roles

Game, UX and interaction design.

Programming, implementation, prototyping.

​

Team

5 members

2 research consultants from QED

​

Timeline

6 Months

(Sept 2019 - April 2020)

​

Tools

Software:

• Unity C#

• Adobe Suite

• Webcam

• Pico Projector

Our challenge

For our final thesis project, my team and I decided to partner with Marcatus QED, which provided our design challenge:

How might we create an interactive experience to teach farmers sustainable farming practices?

Background

With the help of MarcatusQED, we had access to specific research that they conducted and we were able to focus on a specific audience that the design challenge was aimed towards.

​

​

"500 million smallholder farming families produce 70% of the world’s food."

These smallholder farming families can face a lot of challenges and obstacles in taking care of their life's work.

​

• Limited access to formal education.

• Limited access and unfamiliarity to technology, electricity and the internet.

• Language barriers.

• Harsh environmental factors (eg. weather) and life situations.

• Family income is directly tied with produce yield from the farm.

MarcatusQED had made multiple efforts to create video teaching guides using projectors, tablets and other similar projects in efforts to teach smallholder farmers in India about sustainable practices.

These instructional efforts were not as successful as intended since they ran into a number of issues when trying to help farmers switch to more sustainable farming practices.

​

• Switching to sustainable practices requires change, and some farmers can be traditional in decision making

• Switching requires more effort and doesn't always show improvement in yields until a few seasons later. This lack of immediate feedback can make it feel like it's not a good use of their hard work and time.

• Not being aware of how their current practices affect the environment and diminishing long-term yields of their crops makes it an issue that doesn't need to be addressed right away.

• Farmers aren't too engaged just by watching videos, it can feel too detached from their farm work and had a hard time retaining the information presented

Why LightFarm?

Follow a farmer's daily struggles and take care of your farm

By combining exploration and decision-making on an interactive farm, we created a memorable and engaging teaching tool. 

Easy to interact and play with

Our custom-designed controller makes physical interactions feel natural and intuitive.

Easy to learn about sustainable farming

Projection mapping allows us to visualize and convey a large amount of complex information about farming practices.

​

Process

Exploration

We brainstormed and tried a lot of different possible solutions of approaching this challenge to make an informed decision on which direction is best.
 

To help us narrow down on which ideas to pursue, we decided to prototype a wide range of ideas which included AR, MR, VR and mobile implementations.

Creating a vision

After testing a number of prototypes, evaluating them, and considering a lot of tech constraints, we decided on a physical diorama for a number of reasons:
 

• Interesting level design creates an explorative and narrative elements.

• 3D diorama with a cross section helps communicate complex ideas about the soil.

• Physical touch can help make interactions feel better.

Projection Mapping

Since MarcatusQED had a number of projectors available for our use from their other teaching efforts, this gave us a great opportunity to try to use projection mapping as a way to display information on our diorama.

By keeping the diorama generic and ambiguous, we would be able to project different environments and scenarios without having to switch the diorama.

After a few tests with the tech, it become clear that projection mapping fit our project and we decided to pursue it.

Projection Mapping Challenges

As we started testing more complex shapes and builds for our diorama prototypes, some projection mapping challenges became clear:

• Projector has to be facing the diorama from the same general direction as observers to get the best projection experience. Projecting from a diagonal angle can cause the picture to stretch and lose the point of view effect. A solution to this is to stretch it inversely in-game.

• People's bodies and hands often block the projector if they walk and reach around the diorama outside of a specific intended area, creating lots of shadows.​

• ​Projector has to be within 2 meters distance for reasonably high resolution on the projection.

Interactions

To take full advantage of a physical diorama and create really solid interactions, we tested a wide range of input solutions.

Leapmotion seemed to give the best results based on initial testing and we started iterating with this interaction model.

First prototypes

For our first prototype, we created some simple interactions to be able to test our input system.

After our first testing sessions, the feedback became clear that our input system was not consistent enough, missing roughly one third of the touch inputs on the surface. We had to search for a new way to interact with the diorama in a more consistent manner.​

Input rework

Using image recognition software, we found a way to create a touch input on a thin surface using the shadow of your fingertips. This worked great for our solution since the diorama would always be lit up by the projector and the shadows would be easy to detect.

By making a hollow stand and using a combination of clear plastic and paper covers, the webcam at the bottom was able to track from multiple heights and surfaces of the diorama at once.

Testing the new interactions and diorama showed a huge input consistency improvement and players focusing more on the experience.

Product vs Game

It was a challenge to figure out how much content should be gamified and how much should be the content we are trying to teach.

We decided on a linear playthrough with progression based on a series of decisions on how to take care of your farm. These decisions would be focused on a specific lesson and would show you the different pros and cons of sustainable farming practices and how they will affect the farm over longer periods of time.

Game mechanics were designed around this structure to best highlight each decision and outcome.

Outcomes

Takeaways
 

• Lots of quick experimentation is important early on to make informed decisions.

• Defining the boundaries between product and game design and when to use them.

• Importance of space, lighting and view perspective on physical and projection mapping projects.

Awards

Intel 2020 University Games Showcase: Innovation Award Winner.
Rookie Awards 2020: Immersive Game of the Year Winner.
Unity 2020 Awards: Best Student Project Runner Up.
The ADCC: 2020 Student Competition Finalist.
RGD 2020 Awards: Design Award for Story-telling Honorable Mention,  Award for Social Good Design Honorable Mention, Award for Concept Development Honorable Mention