Creating Interactive Learning Tools (or: “Fine, I’ll do it myself”) 

The Program
Written By

The Problem 

I was teaching my Pragmatic Photography course at Carnegie Mellon University. After my lecture on exposure, I switched to the projector and brought up a website: exposuretool.com. Instead of my wonderful interactive example, the website had died, and the domain sold off to some squatter. Whoever had created the project stopped paying for the web hosting, and I was mid-lecture and out-of-luck. 

As an educator, I am constantly looking for ways to engage my students. As an educator in the mushy media and design space near ‘art’ and ‘technology’, I am not without options. What better way to teach media than with media? Virtual reality with virtual reality? Audio with audio? I have a constant need for supporting content for my lectures. While I am something of a savant at gesticulation, hand-waving about signal to noise ratios or visual weight only get me so far. Don’t get me started on my Fourier-transform dance.  

So, to supplement my sign-language wingdings, I rely on interactive teaching aids that help illustrate various points. These teaching aids are the subject of this analysis. As I created and searched for them, I came across a simple problem: what I want does not exist.  

What’s wrong with existing solutions? 

  • Inaccessible. EdTech tools like H5P, or so on share some related issues with accessibility. Notably, anything that requires an account to log into to create, share, or view is a non-starter for me. The tools may be great (they aren’t), but if clicking a link sends you to a login page, it’s unusable during class or after. Anything that is integrated with the LMS becomes inaccessible for review after the class has ended. 

  • Online. If the content for the tool is hosted on a server elsewhere, and can’t be (easily) downloaded or archived, then I can’t trust it. I cannot have my lessons at the whim of some startups AWS payments. I believe that students should be able to access the content in 5, 10, 15 years from now.I consider offline archiving a must-have. 

  • Unwanted Alterations. Many tools are built on SaaS (Software as a Service) companies, and these companies love updating their tools! Great! Except when the UI changes, the user flow changes, or – worst of all – the actual thing you made changes.

  • Not Isolatable. I am not looking for standalone lectures, exhaustive videos, or online courses, but standalone concepts. Making a diagram inside of PowerPoint is less useful than a free-standing diagram that one can put into a PowerPoint, an LMS, a website, or so on. 

  • Gone. I am so tired of fun widgets, websites, and services disappearing from the internet. The Wayback Machine has trouble archiving anything interactive. 

  • Proprietary. Remember Adobe Flash? Or Microsoft Silverlight? Or Adobe Flex? I won’t be making my tools in Roblox anytime soon, thankyouverymuch. 

  • Closed Source. When things are almost what I want, I would love the ability to just copy the software and tweak it for my needs. Why re-invent the wheel? 

All of these observations push me away from SaaS providers, apps, expensive LMS integrations, anything with a ‘freemium’ model, or anything that Microsoft will have me log into.  

Fine, I’ll do it myself. 

After an appropriate amount of grumbling, I realized that it might be easier to make myself exactly what I want. Somewhere on the spectrum between a chalkboard and a virtual reality experience lies the segment of interactive experiences that I am concerned with here. As a web developer, software engineer, and game designer; I have the existing skill set to put prototypes together quickly. This became my tech fellows project.

I prototyped approaches to creating interactive learning tools with a variety of technologies. 

The Process

First, I defined specific objectives of what I want to create.  

  • Interactive, Experiential. What makes a good diagram, insightful gif, or informative representation of quantified information is out of scope. At the least, I want to give students something to play and engage with. A lot of existing tools are for one-directional demonstration, or for when an instructor is guiding the experience. As a game designer, I am biasedtowards experienced with interactive tools. 

  • Flexible, Modular. Creating one of these is a non-trivial amount of work, which deserves getting the most use from it as possible. I want something that can be brought up during lectures without hassle, linked or embedded in an LMS, embedded in a presentation, sent to students in reply to an email, and so on. 

  • Independent, Standalone. Many of my teaching aids are useful during live demos, or illustrative punching bags that I can bop about in front of the class. These are great! But I am also looking for something that does not require expert instructional bopping to be informative. 

  • Not VR. I already work on educational VR/AR experiences. That’s a different project for a different outcome. 

What Content to Cover

Next, I examined how I was handling the problem already. I started with existing lectures that would make for good candidates to be supplemented with an interactive tool. 

  • Photographic Exposure 

    • The dead exposuretool.com was a great tool! I used it! It works! 

    • I should recreate it? but offline and open-source? 

  • Analog vs. Digital (Oscilloscope Lecture) 

    • Difficult for students to see a webcam pointed at a small oscilloscope screen. 

    • Fiddly hardware setup. 

    • One still must connect an intuition of what is being measured to this flat fuzzy line graph. 

    • Hard for students to interact with in a classroom setting, would benefit from 1-1 play, as demonstrated by students who stayed after class to do just that. 

  • UV Maps 

    • Tedious to understand through words. 

    • Prime topic for creating an ‘a-ha’ moment via interactivity. 

    • Doing that interaction in the 3D modeling software is noisily buried in the confusion of the rest of the software. 

  • Perspective Distortion 

    • Lens Width vs. Focal Distance is cumbersome to speak about. It can be demonstrated visually with clarity and ease.  

    • It can be hard to understand on exercise, as one is distracted by the “rest” of photography, cameras, and all those poorly named settings. 

  • Abstract Syntax Tree Explorer 

    • https://astexplorer.net/ doesn’t do a good job surfacing the tree structure 

    • Showing students how an AST works visually is important, but manually diagramming is slow, and in hypothetical-example-land. 

    • Computers are really fast at generating this tree from code. That’s their whole thing! Generating a real-time and snappy visual tree as the student types would be simple and, I think, fun. No need for hypothetical code, they can view their real projects. 

  • Binary, Hex, Positional Notation 

    • After working at a whiteboard for some time, I turn to a group of confused students and say something like “you just have to play with it”. While this is true, it’s not a satisfying educational moment. 

    • I would like to facilitate, directly, this act of students ‘playing with it’. 

    • A topic that comes up a lot, but doesn’t fit into any of my classes well. High need for a standalone “thing”. 

Creating Interactive Tools

Before showcasing the different projects that I created, I’ll overview my technical process and list the tools I found useful. Some may wish to skip this section, and jump down to ‘What I Created’, below. 

Web Development 

For each of the above lectures, I wrote up a design document of the ideal scenario, forgoing pragmatism. Many of the ways we teach are ‘good enough’, and my largest challenge of this project was the line between pragmatic (these are real lectures I must give) and aspirational (I also want them to be good). I have hopes of creating a framework for quickly working, but for my first round of efforts I built each project from scratch and used a different tool each time. In this way, I surveyed different approaches and technologies. I’ll ignore the various false-starts and dead-ends in this blog post. 

I took two approaches: Creating small client-side-only webpages and standalone desktop applications. 

For webpages, all of the interactions had to be client-side in order to stick to my constraints (accessible, archiveable). No node.js! Even more, for any JavaScript frameworks I used, I downloaded and hosted my own copies instead of relying on a content delivery network (CDN). While CDN’s have many advantages, the embedded approach means if someone hits “file, save as” on the page, it will archive everything they need to run the website, forever. 

This meant I had to only use frameworks that can be redistributed under the license I was distributing my projects (MIT). Luckily, this isn’t a problem for nearly all front-end frameworks. 

Some useful ones that helped with interactivity and rendering graphics were: 

As I worked – and brushed off my rusty JavaScript ability – I changed my thinking from trying to find the perfect framework that does as much as possible for me; to ending up on the side of ‘Fine, I’ll do it myself’.  

As it turns out, making websites slightly interactive is exactly the thing that JavaScript is very good at! There is basically no need for much of the complex tools the frameworks provide. What I anticipated would be a challenge ended up being some very intro-level work. It is easier to do simple things with pure JavaScript then it was trying to read Babylon’s documentation.  

I wrote the trivial HTML by hand, and lazily slapped a CSS framework (cutestrap.com) so it doesn’t look terrible. (That’s right, no need for React, Vue, Angular, or even jQuery. You don’t need them!)

GitHub Pages provides free static hosting. Since everything I am making is client side and open source, it’s a great solution and just my budget. I am using my own domain name. If GitHub stops being a viable option, I can change hosts and simply point the domain name to the new place, so I can keep links alive. 

Software Development

I primarily focused on web pages, but sometimes one wants the power and convenience of “the computer you are sitting in front of, which is really good at all sorts of things beyond rendering HTML”. 

My goals were cross-platform (windows and mac, at least) and standalone. Students shouldn’t be asked to install anything. Just an exe or app file you can run, and there it is. 

Unfortunately, software development is not as easy as spinning up quick websites. One solution is to apply web technology to software using tools like Electron. I think these are kind of gross, one of the advantages of writing software is that it doesn’t have the styling pains of web development or take 3 gigabytes of RAM just to open.  

Instead of doing things the easy way, I double-dipped with my professional development goals, and prototyping tools in new frameworks to learn new workflows and languages. 

For the exposure tool, I need hundreds of high-resolution images. Making a website would require hosting these and paying server costs. One day, it won’t be worth it -- a downloadable app makes more sense, as it stays working even long after I have moved on with my life. 

I found the following frameworks useful for rapid prototyping: 

  • Go language and Fyne software framework.

  • Python, various GUI frameworks.

  • C#, various frameworks in .NET ecosystem.

  • RayLib – bindings for various languages including C#, python, go, rust, and lua. Easily create a window and render graphics, text, ui, 3D, etc without the complexities of an entire game-engine. 

  • Godot engine. Using built-in UI tools, GDScript or C#.

  • Unity and Unreal Engine are similarly useful. Godot is lightweight as game engines go.

In general, the web has taken over in terms of “make a quick little thing”, and most robust frameworks and tools (like qt) have pivoted to tailoring to larger and more complex use-cases that, unfortunately, make rapid prototyping a pain. 

There are some interesting future opportunities to build for Web Assembly (WASM), and write once in, say, Rust, and compile for both desktop and web. WASM is exciting but still too new to feel comfortable utilizing. 

My favorite solution for rapid tiny software development is the game engine Godot. It's open-source and has a comprehensive UI framework. It can build for windows, mac, and Linux.

At the end of the day, there are a lot of options, and one should just work in the framework or language that they enjoy working in. So long as that isn’t Java. 

What I Created 

What Are UV Maps? Interactive Explainer 

The first lecture I tackled was UV maps: 

https://explainers.hdyar.com/uvmaps/index.html

I created a webpage version of a lecture I give, turning slides into embedded gifs or video snippets. I looked at the places where, previously, I had jumped over into software, and tried to identify the single piece I was attempting to illustrate. With this in hand, I worked on creating small interactive javascript widgets to explain each piece. 

/

Hover your mouse over the embed above.

This sampling demo was the hardest to create, and the first and last time I will likely use the BabylonJS engine. 

I assigned the page to students as homework, then assessed them the next week in class as I watched them work on their 3D model projects. The result? Success! While the effort to create the page was high, as a standalone assignment/lecture activity, it works very well. 

Perspective Distortion

For the next project, I focused more on smaller, single-concept interactions, and on more pure JavaScript. 

https://explainers.hdyar.com/perspective/index.html

I took my perspective distortion lecture and wrote it out. It starts by showing a video of the Hitchcock Zoom, and I can just embed that video in the page.  

Then, the two points I emphasize with interactions:

https://explainers.hdyar.com/standalone/fov-distance.htmlhttps://explainers.hdyar.com/standalone/camerasubjectbackground.html

I have not yet assessed how the page works standalone, as I use it during a lecture, bringing up just the interactive toys and sliding them while lecturing over top. The students are given a link to the full page for review. Since it is combined with an in-person lecture, I do not have feedback on how it works standalone. For an in-person lecture, it works far better than my (awful) whiteboard drawings. 

The subtitle of this project might be “how far will I go to prevent students from discovering how bad I am at drawing on a whiteboard”

Binary, Hex, and Encoding Numbers 

For this content, I knew I needed something I could send to students who needed a sort of “review+fleshing out”, where they circle back on a concept they have sort of seen, or encountered on the periphery, and then center on it. It’s never something I directly cover in a lecture or class, but comes up all the time. 

I first created a video covering binary. This wasn’t quite enough, so I planned out a fuller video on positional notation, and then stitched it all together in a webpage.  

The video content is designed to be encountered in the page where it is. So we have: 

  • Written content 

  • Video content 

  • Interactive tools 

  • Real-life examples 

All living around this web page as an entry-point: 

https://explainers.hdyar.com/binary/index.html

The interactive parts of this page were the easiest to create. Less than 100 lines of good-old-fashioned pure JavaScript (https://github.com/hunterdyar/explainers/blob/master/binary/binary.js), no graphics rendering, or frameworks involved. The biggest challenge was writing a roman numeral converter, and if you think I didn’t just dive to stack overflow to use someone else’s vetted code, you are sorely overestimating me. 

These tools work well because they are so small.

https://explainers.hdyar.com/standalone/binarycounter.html

I have already used them, pulled up full-screen, when giving the lecture in person.

https://explainers.hdyar.com/standalone/color.html

Exposure Tool 

For this, I recreated the exposure tool website, but as a downloadable tool:

https://github.com/hunterdyar/exposure-tool-desktop-godot/releases/tag/0.01a

For an exposure tool replacement, I first tasked a student worker with taking photos at every exposure setting. 20 gigabytes of data later, I had to get it usable in some software.

I used a Photoshop batch process to convert all of the RAW images into smaller jpegs. Then I wrote a python script to automatically rename the files based on their EXIF metadata. So a photo captured at a 1/30s, f11 and ISO 800 became "s30_a11_iso800.jpg". This meant I didn't need to catalog or track all of the files during capture, but could search by filename.

After tinkering in Go/Fyne, I settled on Godot and created the actual software application to view the images in an afternoon.

Lessons Learned 

I think the ultimate lesson I learned was to stop searching for an existing solution to my problems, and to trust my own ability to create. As an educator in media, art, and technology; this is constantly a point I attempt to drive home to students. To solve the problem, prototyping became the perfect solution. 

Throughout this entire project, I spent more time searching, researching, reading documentation, and testing solutions than I did creating my own - by far.

The struggle comes from not understanding scope. The unknown is an unknown scope! Maybe it’s a lot of work! The known, no matter how inefficient or slow it may be, is visualizable. One can map out a path and plan around it. Now I have a solid idea of just how much work it will be to create another little JavaScript widget where a diagram changes as I move sliders around. Turns out: not that much! 

Thanks to the past year of focusing on prototyping, I can begin to find more opportunities to create these tools and embed them where appropriate. I still have long-term goals of my own framework or utility scripts that allow me to work even faster, but I have gained the confidence.

How To Create a Little Thing, Quickly 

With part of this project overlapping my own professional development goals, I can’t recommend everyone take the same approach that I did. Using a new creative framework for every project is not viable unless learning how they all work is a goal of yours, as it is mine.  

So what should you do? 

I recommend heading to processing! P5.js in particular. P5 is a port of the processing creative coding tool (originally written for java, bleh) for the web. It has a web-based editor, tons of examples and educational resources, and it’s extremely easy to pick up. In fact, when I teach programming to new students, it’s one of the “entry points” I recommend. So allow me to recommend it to you, someone who may or may not have any experience in coding. Coding for visual outputs is fun and rewarding, and – oh yeah – it happens to align with our goals of creating little interactive diagrams, that you can share as easily as with a link.  

Even if you have never programmed before, I believe most anyone can get started tinkering in P5 and make something interactive within an hour. One only needs the ability to type, and middle-school level pre-algebra. 

Sometimes it's easier to just do it yourself.

Previous

Enhancing Critical Thinking in Athletic Training: Developing Interactive Simulations for Student Evaluation
Next

Integrating Video Software for Clinical Role Plays