I've been thinking a lot lately about how to correctly calibrate my
various light meters. I have several, of both the incident/lumisphere and reflective/spot types, and they're all over the place. Not only do I want them to match, I'd also like them to all give the "correct" reading based on standards and specifications.
The Internet is chock full of people sharing
their rules-of-thumb and get-it-in-the-ballpark methods, but pretty much none
of those even vaguely resemble what a professional might do. The
closest you'll ever get to that, is recommendations for specific labs that
can do calibration.
What no one ever talks about, is how those labs actually do it.
That being said, I've recently decided to tackle the problem myself. While I'm never going to match the setup of a full-blown metrology lab, I should at least be able to attempt a somewhat calculated approach.
The Printalyzer Densitometer project is my attempt at building a new budget-friendly densitometer to make available to the resurging analog photography community. My ultimate goal with this project is to eliminate the cost and convenience barriers that currently stand in the way of many, who might want to try using such a device.
Its actually a project idea I've been kicking around for a long time,
but initially sidelined due to concerns over optics and calibration.
However, I've recently become motivated to revisit the idea.
Since I wrote my original introductory blog post on this project, I've made considerable progress. The first prototype is now fully constructed, and I'm making significant headway in the development of the firmware. As such, I think its time for another project update.
Given that the project is being developed entirely as open-source hardware and software, the latest schematics and code can always be found in the project's Github repository.
Main Unit Construction
The main unit consists of a rather large PCB, with a double-sided design. The reason it is large is that all the components and most of the ports of the Printalyzer are directly mounted to the board. The way I decided to do this, the board is actually mounted in the enclosure upside-down. This puts the buttons and display on the "bottom", while everything else is on the "top". I took this approach because several of the components are rather tall, and there wouldn't be enough clearance if they had to fit within the height limits of the buttons. From my teardowns of other pieces of darkroom electronics, this construction approach is actually quite common.
For the enclosure, I designed something using Front Panel Express and their housing profile script. While rather expensive for building in quantity, at the prototype stage this gave me very nice results.
For the top of the unit, I installed a piece of Rosco Supergel #19 (Fire Red) sandwiched between some transparent material in the gap between the top of the display and the enclosure. This both protects the display and keeps its output spectrum paper-safe.
Finding a nice metal knob for the encoder, without a pesky direction-mark, took some effort. Ultimately I went with a type of knurled guitar knob (from mxuteuk), which seems to be the magic set of keywords to find these things.
For the keycaps, I ultimately went with black. This was mostly because it was the only color for which I had all the necessary parts at the time, given that these are often backordered or special order with lead time. When looking at the unit on the desk, black also seems like the nicest choice. However, in the darkroom, they are hard to see. I also have these keycaps in dark gray, and they can be special-ordered in light gray. (Having that array of LEDs around the keys really helps, of course.)
Meter Probe Construction
The meter probe is a much simpler device. It essentially consists of a light sensor, a button, and minimalist support circuitry to handle power regulation and voltage level shifting. Right now the sensor I'm using is the TCS3472, which has well-documented lux and color temperature calculation guides and might be useful for color analysis as well.
(One benefit of my design here, is that I can make future versions of this meter probe using basically any sensor I want. The only requirement is that the sensor, or really just the front-end of this circuit board, has an I2C interface.)
Since the enclosure requires a somewhat custom shape, I had to go with 3D printing for this one. For the two prototypes I've built so far, one was self-printed and the other I had printed by Shapeways. The later looks a lot nicer, so that's what I'm showing here.
The hardest part of assembling this was probably the fiddly process of connecting the cable. For the cabling, I went with a 6-pin Mini-DIN connector. In practice, that meant finding someone selling M-M black PS/2 cables and slicing one in half.
One last thing this meter probe needs, is some sort of cover for the sensor hole. My plan is to have something printed/cut for me out of something like polycarbonate, from a place that makes panel graphic overlays. Ideally this overlay will be light in color, with markings pointing to the sensor hole. Thus far I've been putting off this part, but getting it in place is going to be essential to finalizing my calibration of the sensor readings.
Firmware Development
I'm now at the stage in firmware development where I have all the hardware inside the Printalyzer working, and have a stable base from which to build up the rest of the device's functionality. My initial goal here is to fully flesh out the functionality of a mostly-full-featured f-stop enlarger timer, then to later follow on with the exposure metering functions. I'm going about it this way because metering requires a lot of testing that's hard to do "outside the darkroom," and I'll need the timing functions regardless.
(In all the screenshots below, I apologize for the "gunky" picture quality. The red gel sandwich over the display tends to reflect a fair bit of dust, which ends up looking far more distracting in pictures than it does in real life.)
Home Display
The home display shows the current time and selected contrast grade. It also has a reserved space for the tone graph. While I have figured out what that graph will look like, its not easy to show here until I do actual metering.
From this display its also possible to change the adjustment increment, and obviously to start the exposure timer.
Another feature of the home display (not shown here) is fine adjustment. Using the encoder knob you can enter an explicit stop adjustment (in 1/12th stop units), or even an explicit exposure time. I expect this capability to be useful for repeating previous exposures, in lieu of re-metering.
Test Strip Mode
The test strip mode supports both incremental and separate exposures for test strips, and making strips in both 7-patch and 5-patch layouts. The design places the current exposure time in the middle, and then makes patches above and below this time in units of the current adjustment increment.
Another nice feature, made possible by having a graphical display, is that it actually tells you what patches should be covered for the next exposure. This way its harder to lose your place during test strip creation.
Settings Menu
The settings menu is fairly rough at the moment, and probably does need some user interface improvements. That being said, it shows another benefit of using a reconfigurable graphical display. I can show nice text menus that actually tell you what each setting is, and then return to a full-screen numerical time display when done.
Enlarger Calibration
This little tidbit is the first feature I've worked on so far that actually uses the sensor in the meter probe. Its not yet complete, from a user-interface and settings point of view, but the underlying process is fully implemented now.
The basic issue is that any enlarger timer is effectively just "flipping a switch." It can control when that switch turns on, and when it turns off, but little else. Unfortunately, lamps typically do not turn on or off instantly. On both ends, there is a delay and a ramp time. While these are typically short, not accounting for them can cause consistency issues with short exposures. This problem becomes especially troubling when doing incremental short exposures, such as with test strips and burning.
The goal of the calibration process is to make sure that the "user visible exposure time" is the time the paper is exposed to the equivalent of the full light output of the enlarger, rather than simply the time between the on and off states of its "power switch."
Calibration works by first measuring reference points with both the enlarger on (full brightness) and the enlarger off (full darkness). It then runs the enlarger through a series of on/off cycles while taking frequent measurements. When complete, it generates 6 different values:
Time from "power on" until the light level starts to increase
Time it takes the enlarger to reach full brightness (rise time)
Full brightness time required for an exposure equivalent to what was sensed during the rise time
Time from "power off" until the light level starts to decrease
Time it takes the enlarger to become completely off (fall time)
Full brightness time required for an exposure equivalent to what was sensed during the fall time
These values are then fed into the exposure timer code, and used to schedule the various events (on, off, tick beeps, displayed time, etc) that occur during exposure.
Using the above numbers as a rough example of what this all means, without profiling a simple 2 second exposure would actually expose the paper to light for about 2.4 seconds but only a 1.88 second equivalent exposure.
These numbers come from a simple halogen desk lamp, which is more convenient for basic testing. I've also run the same calibration cycle on my real enlarger. It takes longer to rise to full brightness, but is faster at turning off. (Its similar case is 2.2 seconds with a 1.78 second equivalent exposure.)
Now I'll admit this doesn't seem like much, but enlargers do vary and it
can suddenly become critical if you're making <1s incremental
exposures for test strips or burning.
Blackout Mode
This feature is a little "quality of life" nicety that I haven't seen anyone else do. Every once in a while, some of us want to print on materials, such as RA-4 color paper, that has to be handled in absolute darkness. This means once your enlarger is all setup for the print, you need to turn out all of the lights in the darkroom before taking out the paper. Having to manually turn off your safelights and throw a cover over your illuminated enlarger timer, gets a bit annoying.
This is why I added the "Blackout" switch to the Printalyzer! Flipping this switch turns off the safelight and all of the light-emitting parts of the device. Of course everything still works when in this mode, so you can still make prints and test strips in complete darkness. (Eventually I'll probably add additional audio cues and lock some ancillary features to improve usability.)
Next Steps
At the outset of this project, I collected a long list of "good ideas" for things the device should do. That list is still growing, of course. Eventually I will get to them, but right now my priority is to get the standard features implemented and stable.
I have a few more things to do for the basic f-stop timer functionality, such as burn exposure programs. After that, I need to work on an actual user interface for adding/changing/managing enlarger profiles.
Once these things are done, I'll finally move on to paper profiles and print exposure metering. I expect that to be a long-tail item, because it will require a lot of work on sensor calibration and learning about sensor behaviors. However, I plan to do as much this as possible via first converting raw sensor readings into stable and standardized exposure units. This way, it will be possible to experiment with different sensors (as needed) without dramatic changes to how the metering/profiling process works.
Another thing I'll likely do interspersed with this, is start to take advantage of the USB port I put onto the device. There are a few features I want to implement with this, including:
Backup/Restore of user settings and profiles with a thumbdrive
Firmware updates with a thumbdrive (right now it requires a special programmer device, which isn't practical for "end user" use).
Keyboard entry of profile names (I'll make sure they can be entered without a keyboard, but being able to use one is a nice thing.)
Connection to densitometers to help with paper profiling (not sure how this will work, but it can't hurt to try).
I'm still not sure whether, or at what point, I'll proceed with transforming this project into an actual "product" or "kit." The biggest hangup is really that the device necessarily plugs into mains power and switches mains-powered devices. That means it may be hard to safely sell/distribute units without first forming a company and going through various painful and expensive certification processes. However, I'm not going to worry too much about that until I have something rather complete.
Regardless, all the necessary data to build one yourself will always be freely and publicly available. However, actually assembling one of these does require circuit board assembly skills and tools everyone isn't likely to have.
The Printalyzer is a new project I've been working on for several weeks now. It aims to become a modern full-featured darkroom enlarger timer and exposure meter.
What is it?
At its core, this is a timer for a darkroom enlarger. That means it has a switchable outlet on the back, and can turn an enlarger on and off for a set amount of time. However, it is going to be far more sophisticated than a simple clock. Its going to allow the adjustment of exposure time in "stops," making it what is known as an "f-stop timer." That means you adjust time in logarithmic units of exposure, rather than linear increments. Here's an article that attempts to explain the concept.
Of course it'll also include features to help with making test strips, and calculating dodge/burn exposures in stop units as well.
In addition to all this, the device is going to include an exposure metering probe. This probe will let you measure and visualize the contrast range of a negative before printing it. The idea is that you can figure out a fairly decent choice of both contrast grade and exposure time, without even making a single test strip. Of course you still can make the strip, but it becomes more about fine-tuning than making your initial decisions.
Why are you doing it?
Why have I chosen to take on this project? Don't similar devices already exist? Well, technically they do. Several, in fact. You can find everything from polished commercial products to crowdfunding campaigns to homebrew hobbyist projects.
Okay then, why am I bothering?
Well, in short, I've found myself growing increasingly frustrated with what is currently out there.
Most of the commercial products in this space are quite expensive, and were designed and built in the mid-to-late 1990's. While they do work, they tend to suffer from all the limitations of "built to cost" embedded devices from that era. This often means a limited user interface that can be difficult to use without constantly referencing a manual, firmware that is difficult or impossible to update, features that are constrained by the capabilities of an 8-bit microcontroller, fixed peripheral choices, and a general lack of new development work.
Many of the more modern projects tend to forget things that made those old devices great, like a sturdy case, real buttons, and a no-nonsense primary interface. They also sometimes try to do too much, such as attempting to be a general purpose darkroom timer. Finally, its rare that they tackle the exposure metering problem.
What are your project goals?
Basically, to take what I like from those older products and to bring it up to date with more modern embedded technology.
Fundamentally, I like to think of this project as building a platform, rather than a static appliance. Sure, its going to have all the necessary hardware to be an enlarger time and meter. However, its also going to have enough excess capacity so that it can be continuously updated to increase its functionality.
My goals for the Printalyzer include:
Use real buttons for the user interface, and have enough of them that the need for awkward combinations is minimized. A rotary encoder knob may be used where it makes sense, but not as a catch-all.
Use a graphical display whose layout can be changed depending on the device mode. It can emulate the look of 7-segment and bar-graph LEDs when that makes sense, but it can also display nice menus for setting things up.
Use a flexible interface for connecting the metering probe, such that the choice of light sensor isn't baked into the device. It should be possible to use different sensors as desired, simply by plugging in a different probe. While I'll initially focus on B&W metering, being able to function as a color analyzer is absolutely a stretch goal.
Have enough program memory that all desired features can be implemented. There should never be a reason to need different versions of the device to offer different feature sets.
Have enough user memory that settings and calibration profiles are not arbitrarily limited, and can even be accompanied with meaningful descriptions.
Include a USB port, so that settings can be saved/loaded from thumbdrives. It should also be possible to use this port to connect to other peripherals. This could include keyboards, for typing profile names, or even densitometers to automate the process of creating profiles.
Include a "Blackout" switch that turns off all illumination on the device, enabling the user to perform basic functions by relying entirely on audio cues. This will be of great benefit when doing color printing, where paper must be handled in total darkness.
Able to run off all common mains voltages, without modification (except maybe a different fuse).
All hardware and software for the device will be completely open source, so that anyone can build or modify it.
My anti-goals, or things I explicitly do not plan to include:
It will not be a multipurpose darkroom process timer. That means no timing of film development. This timer is meant to sit next to the enlarger, not to be carried around and used for everything.
It will not require any sort of smartphone app or computer interface to set this thing up. It should be possible to use the device as a completely standalone unit, where all of its capabilities should be locally configurable.
How will it work?
The device will consist of a main unit that looks somewhat like every other enlarger timer, and a connected metering probe.
Here's a basic block diagram that shows the main components:
The main unit has buttons, a display, illumination LEDs, a buzzer, and relays to control both the enlarger and the safelights. The metering probe has a button, to trigger readings, and a light sensor.
How far along is the development?
The first revision of all the schematics and PCB layouts are finished. I've constructed the first prototypes of the metering probe, and am in the process of constructing the first prototype of the main unit. Once that is done, I'll have to do a lot of testing and begin to write the software. Since this is all open-source, you can see the nitty-gritty of this work on Github.
As far as the theory and process of print exposure adjustment and metering, I've also done a lot of research to prepare for the project. In addition to what I can figure out on my own, it also helps that there are a lot of long-expired patents that go into a good amount of relevant detail on the subject. Additionally, I'm hoping I can make the calibration profiles for this device compatible with other existing devices, to make initial setup as easy as possible.
Will I be able to get one?
That is the hope. While the first prototypes are mostly for me to be able to tinker around with an enlarger timer/meter that I can reprogram myself, I would eventually like to make this into a real product. I'm not yet sure if it'll be a DIY thing or a complete constructed unit. Its also likely that there will be non-trival legal, regulatory, and manufacturing hurdles to make this happen. I'll also need to get the cost down, as nice-looking prototype enclosures can be quite expensive.
When I designed the main board of the Nestronic, I was busy enough that I didn't want to hold up progress on account of other areas of the project. I basically just figured out what interface pins I might need for an input board, brought them out to a connector, and set that part aside.
I figured I'd need an I2C bus to interface with some sort of I/O expansion chip, and a few support GPIO pins. I also thought it might be neat to try experimenting with the ESP32's touch sensor capabilities, so I made sure at least one of the GPIO pins could be used for that.
I eventually got to the point where the schematic was mostly finalized and the whole thing was working on a breadboard. While a big achievement in and of itself, it was far from a finished product.
First, breadboards usually look kinda messy. After all, they tend to evolve as you tinker with the circuit design. Also, breadboard wires are never the exact right length for any given connection. While you can make a clean-looking breadboard, it often requires far more time and effort than one is willing to put in for a simple prototype.
Second, breadboard connections are a little flaky. Breathing on your circuit the wrong way can cause this wire or that resistor to have just enough of a contact issue that something starts or stops working reliably.
Third, breadboards are noisy and full of stray capacitance. I could detect clock signals from one end to the other. Digital connections sometimes had just enough noise to make the difference between a high and a low. And to quote Dave Jones, you sometimes have to "hold your tongue at the right angle" to get things to even start up correctly.
Breadboard Nestronic
At this point I exported the netlist for my schematic, pulled it into KiCad's PCB tool, and began the layout and routing process. After a lot of work, I eventually ended up with a complete board design. The core of this layout was the NES CPU section, with all its parallel bus traces. The rest was kinda squeezed in around it. Perhaps not the most optimal of layouts, but good for the general size and shape of the enclosure I wanted to put it in.
(For an introduction to this project, please read the previous blog post.)
System Design
Now that the design of the system is mostly complete, I'd like to present it in a little more of a top-down fashion. First, I'll present a block diagram of the complete system. Then I'll go in-depth on how I arrived at all the relevant elements of the system. Finally, I'll show the detailed circuit schematics this all represents.
The design of the Nestronic consists of two major subsystems. First, there's the NES CPU Section, which builds around the RP2A03 CPU and contains everything necessary to actually synthesize video game music. Second, there's the ESP32 Microcontroller Section, which provided the front-end and completes the project.
After finishing my previous project, I felt I needed something new to keep up the momentum. There were a lot of things I learned about during the development of the project, which I didn't use but but wanted to explore. I also liked the idea of projects that combined modern and retro elements.
I spent a lot of time scouring the Internet for ideas, but was frustrated with what I found. Far too many projects seemed to be rehashes of the same old ideas, often involving interfacing someone's favorite microcontroller to some sensor or display device. Plenty of elements that would be fun to use as part of a project, but nothing too inspiring on its own. In other words, I really didn't want to build yet another weather station.
One idea that did mildly pique my interest was that of a "modernized" alarm clock (kinda like this). You know, one that could actually sync its time via NTP and that wouldn't lose minutes if left unplugged for too long. I wasn't sure if this idea was tempting enough on its own, but perhaps it could be combined with something else...
As the search wore on, I eventually stumbled upon a project that really caught my eye. This project was Aiden Lawrence's Sega Genesis Music Player. It took the original synthesizer chips from the Sega Genesis, combined them with a modern microcontroller, and used all this to "authentically" play video game music. This idea was almost perfect!
Okay, I say almost, because I didn't actually have a Sega Genesis when I was growing up. Instead, I had a Nintendo Entertainment System (NES). So what if I could build a similar project around the NES? What would it take? And could I combine it with my previous idea to make the ultimate "Nintendo Alarm Clock"?
Unfortunately, unlike the Sega Genesis, the NES does not actually use separate audio chips. Instead, its CPU is actually a modified 6502 core combined with an integrated audio processing unit (APU). As such, I'd need a fundamentally different design. Instead of simply interfacing with audio chips, I'd have to actually build a system capable of running code on that CPU. Thus began a lot of research, including reading all the NES and Famicom schematics posted online. I also found the resources from the NESizer2 project to be quite valuable.
So I finally had my idea! Build a device that combined a NES game music player with an alarm clock! I could wake up to various video game soundtracks, and even get cool sound effects when pushing the buttons.
Initial Approach and Project Goals
The first thing I'd need for this project, is a way to actually run code on a NES CPU from within my own system. This would make it possible for the NES CPU to communicate with a data source and poke values into its audio registers.
There were two ways I could go about accomplishing this:
The hack'ish way involved a modern microcontroller messing with (or pretending to be) the memory on the CPU's bus. (The NESizer2 implemented something like this.) This approach certainly required fewer components, but would involve chasing clock timings. It also seemed a little bit too quick-and-dirty for my tastes.
The more conventional way, which I ultimately opted for, involved putting an SRAM, EEPROM, address decoding, and I/O peripheral ICs onto the NES CPU's bus. Doing this, I'd then write a complete program in 6502 assembly code to transform the NES CPU into an externally controllable audio synthesizer.
The second thing I'd need was a modern microcontroller to act as the front-end. I decided to use this as an excuse to play with the Espressif ESP32, which I had learned about during the development of my last project (which used an ESP8266).
Finally I'd need a bunch of peripherals... A nice display, an SD card interface to provide access to data, some input devices, etc.
At this point, I set out a number of basic goals for the project:
Use an authentic NES CPU (RP2A03) to synthesize audio
Use entirely modern components (beyond the NES CPU)
Finally embrace surface mount components, which I'd explicitly avoided in all my previous projects
Music Source
At the outset of this project, the VGM (Video Game Music) format seemed like an ideal thing
to start with. Its what Aiden's Sega Genesis project used. There were a lot of game rips already available, and the format was
literally "poke this value into that register, sleep for X samples, poke
this value into that register, etc." It seemed almost trivial to write code that could transform a VGM file into actual NES APU writes.
As
I got deeper into the project, I also learned about NSF (NES Sound Format). It seems like NSF has
an even larger catalogue of rips and is more popular with the chiptune
community. Unfortunately, NSF seems to only be viable on emulators (or an architecturally correct NES with whichever bank switching chip a particular game felt like using). So
when/if I decide to start dealing with NSF files, I think I'm going
to have to basically convert them to VGM first. (Either by hacking an
emulator on my PC, or by running a stripped down emulator on the ESP32
that captures APU writes and sends them across.)
Early Tinkering
I began my actual work on the project by proving out various elements of the NES portion of the system. The first step was to get myself an SRAM chip, an EEPROM chip, some 74-series glue logic, and actually build myself a functional "computer" around the RP2A03.
Breadboard NES Core
To prove that I could actually execute my own code on this thing, I fired up my old HP 1630G logic analyzer. It was of similar vintage as the NES itself, which somehow felt appropriate:
HP 1630G Front Display
I felt a huge sense of victory when I finally got to the point where I could write code, compile it, put it on the EEPROM, and see the RP2A03 actually executing it on the logic analyzer:
Logic Analyzer Probes
Program Code Execution
At this point, I duplicated the audio output components from the original NES schematic, wired them up to a pair of amplified computer speakers, and began running some APU test code that I found online:
Since everything was still working, I decided to move onto the next step.
I read up on the VGM format specification, and began writing a C program to parse the NES-specific subset of it. Testing with the theme of a famous video game, I started by trying to see if I could programmatically convert that theme's VGM data directly to 6502 assembly for the NES CPU. It actually wasn't that hard at all! But it was big! Trimming it down to fit within my paltry 8KB EEPROM only got me a few seconds!
But these were a glorious few seconds of victory!
I then modified my C program to produce output in a more compact binary form, and modified my 6502 assembly program to iterate over this binary data and "play" it. This made room for a few more seconds of song, and gave me a more useful test-bed to evaluate some ideas for how to actually feed data into the system during the next phase of the project.
I also built out a basic audio amplifier circuit, using an LM386 and a small 8
ohm speaker:
Thus, the final proof of concept!
Next Steps
The next step is to take what has been accomplished so far, and to build a complete system design around it. That involves selecting a display device, determining how best to configure and use the ESP32 as a front-end, and most importantly, figuring out exactly how I'm going to make the ESP32 feed data into the NES CPU. I'll be detailing this in the next blog post.
Afterward, I'll need to design an enclosure, control pad, and write plenty of software.
This post is definitely not going to completely describe the software. The software is always going to be a work-in-progress, so follow the Git repository for more specific updates. Instead, its going to describe a bit about the development setup, and some basics about what the software is doing.
The first step in this stage of the project was to actually have a development board to work with. While I could have simply used my real circuit board (as described in the previous post), doing so had a number of disadvantages:
Somewhat tedious to switch in-and-out of flash download mode
Required an extra adapter to connect to USB
Designed to be powered by a bulky transformer connected to AC mains
No way to test the coin switch outputs without a multimeter
No way to test the pulse decoding input without a real wallbox
In other words, using the real circuit board was not a solution well suited to an office desk environment that was outside of my electronics workbench.
To do the majority of the software development, I really only needed to test the ESP-12S itself and basic interactions on its I/O pins. Thankfully, this did not actually require the full circuit board. So, to make my life easier, I cobbled together a little breadboard development rig.
Development Rig Breadboard
For the ESP-12S part of this rig, I used an Adafruit Feather HUZZAH. Its a convenient breakout board that includes a USB port, reset button, and some nifty wiring that allows the USB serial controller to automatically switch the ESP-12S in and out of flash download startup mode.
To simulate the signal pulses of the wallbox, I used an Arduino Nano. This module also has its own USB port, and I/O pins that could easily be programmed to simulate any digital pulse stream I needed. Since I'm not testing the full filtering path, I really only need to simulate a clean series of logic-level pulses. The code that runs on this Arduino can be found here: [PulseSim.ino]
For the coin switches, three LEDs in different colors were enough. After all, I just needed to test that I could make them blink.
Since the Arduino is a 5V system, and the ESP8266 is a 3.3V system, I leveraged my existing bin of components to link these together. I used a spare opto-isolator and inverter, which did the trick. This had the benefit of making the two microcontroller modules completely isolated from each other, so they could be powered independently.
In schematic form, it looked like this:
Development Rig Schematic
Setting up the toolchain
The easiest way to get the development tools up and running was to leverage the "esp-open-sdk" project. This project is essentially a Makefile and some scripts that downloads, compiles, and configures everything you need to write code for the ESP8266. This part was surprisingly painless, and decently covered in that project's README.
Decoding Pulses (For Real)
When I was originally figuring out the pulse protocol and filtering circuitry, I used an Arduino-based test decoder. The code for this basically used the "pulseIn()" function in a loop. Upon further investigation, I found that "pulseIn()" just busy-waits on the I/O pin. While perfectly fine for basic testing, this was definitely not a good solution for a complete system.
The approach I took here involved configuring an interrupt to trigger on any transition on the signal pulse's input pin. Inside the interrupt handler function, I then populated an array with the pulse durations and pulse gap widths and kicked off a timer. If no pulses happened for a certain amount of time, the timer callback function would then inspect the array and determine what the song selection was. This approach ended up working quite well.
While precise timing is really not necessary for the wallbox's coin switch mechanism, I still went and measured how long a real coin would actually trip the switches for (40-60ms). I then wrote some simple timer functions that would trigger the selected coin switch for the appropriate amount of time.
This is the step where things started to get complicated. Some of the blog posts I read at the outset of this project had led me to believe it was just a matter of spitting the right HTTP POST at a Sonos speaker's IP address, but is anything actually that simple? Especially if you want a robust and reliable solution?
As I worked through this process, I had decided that there were certain things I definitely wanted to have:
Sonos device configuration should be easy (i.e. no hard-coded IP addresses)
Can't make any assumptions as to the state of the Sonos speaker
The Sonos device's playlist should be treated as if it was a jukebox
The goal was not only to make setup easy, but to also ensure that I should never have to fiddle the Sonos app on my phone to get things into (or back into) a functional state. The Wall-O-Matic interface should function as standalone as technically possible.
Of course doing all of this meant that I needed to figure out quite a bit more of the Sonos protocol than a single enqueue request.
To figure all this out, I spent a lot of time staring at Wireshark while fiddling with both the official Sonos app and the node-sonos-http-api project's utility commands.
I'm not going to even attempt to document the Sonos protocols here, as that would take way too much time and effort to write up. I'll just be giving a short summary of the parts I used, then linking to my code that implements them.
Discovery Protocol
The Sonos discovery protocol is based on SSDP, and involves two kinds of device discovery: explicit searches and notifications.
Explicit searches begin by sending a UDP broadcast to IP address 239.255.255.250 (port 1900) with the following payload:
Additionally, Sonos devices will periodically broadcast their presence on port 1900 without any discovery requests. When that happens, the payload looks similar:
Regardless of how it came in, I collected the IP address, port, and UUID from these payloads. Then, to get the zone name itself, I made an HTTP GET to the following URL:
http://192.168.1.42:1400/status/zp
The response to this was a big XML blob that contained the zone name, as well as other device information.
This info was then collected in a data structure, and later used as follows:
IP and Port - Used to actually connect to the device and to send it commands
UUID - Used as the reference to the device kept in the software configuration, and used as part of certain control requests.
Zone Name - Used for display purposes, so the UI could give friendly information on which zone was selected.
To instruct the Sonos device to actually do things (e.g. add song to queue, play, query status, etc), I had to implement the following control requests:
AddURIToQueue - Adds a file or stream URI to the playback queue
GetPositionInfo - Gets position info on the current track
SetAVTransportURI - Sets the transport URI to play (e.g. stream source or local file queue)
Unfortunately, the normal control interface provides no way to distinguish between the playing and paused states. The only way to do this, was to listen for event notifications. There's another protocol Sonos devices support where you can tell them to notify you whenever there's a state change. Thankfully, these notifications include the play/paused state.
At boot, the device discovery listener is started and a discovery request is broadcast. As soon as the configured Sonos device UUID is detected, it is set as the active device and an event subscription request is made.
When a song selection is made on the wallbox, the following sequence of operations are initiated:
Construct a track URI for the selected song
Add the track URI to the Sonos device queue (AddURIToQueue)
Get the current position info (GetPositionInfo)
If the current transport is not a file URI, then switch to the local queue (SetAVTransportURI)
If the current transport is a file URI, and currently playing, then do nothing
Otherwise:
If not currently playing, start playing
If on a previous track, and not currently playing, then seek to the added track and start playing
If on a previous track, and paused, then seek to the next (or added) track and start playing.
At runtime, the wallbox itself kinda is the user interface for the project. Well, with one exception: inserting coins. I needed some way to instruct my board to "insert a nickel/dime/quarter" remotely, either via a manual interaction (i.e. click a button on an app/website) or via some sort of home automation setup. To accomplish this, I decided I'd simply expose some sort of basic HTTP API for triggering the coin switch relays.
At setup time, it was a different story. There was actually a lot to configure for this thing to work correctly:
Wi-Fi Network Setup
Selecting the Sonos device to play through
Selecting the type of the connected wallbox
Configuring which song file to play for each wallbox selection code
Some might simply bake all this into the firmware. I really didn't want to do this, since you'd never design a "real" product like that.
So the first thing I did, was to embed a web server into the device. To do this, I leveraged the libesphttpd project (and its sample esphttpd app). This project provides a simple webserver written against the ESP APIs, and a fair amount of sample/utility code to handle support functions (Wi-Fi setup and firmware updating) that I knew I was going to need.
As I began to explore this project, I soon noticed that it hadn't been updated in a while. However, there seemed to be a number of forks that had been picking up improvements and fixes. Ultimately, I settled on using Chris Morgan's fork of the project.
After several iterations, I ended up with a (mobile friendly) landing page that looked like this:
Local Web Interface
The setup and firmware pages were pretty much straight lifts from the esphttpd sample project, albeit with some minor style tweaks. The Sonos zone selection page was also fairly simple. The most complex of these pages was the "Wallbox Configuration" page:
Wallbox Configuration Page
The "wallbox type" determines how many song selections there are, how they're numbered, and what the format of the wallbox's signal pulses are. (Contrary to a previous post, I did eventually get the model "200" mostly working and added code to support its signal pulse format. The format is different from the "100", and actually slightly simpler.)
The (long) "base folder path" and the (short) "song" filenames are combined to form the playback URI for each song selection. Buttons help automate the otherwise-tedious entry process.
The layout of this page reflects the approach I took towards configuring the playback URIs. While I could have allowed a completely custom per-song-selection URI, doing so would have required quite a lot of configuration memory.
The ESP SDK has APIs for "safe" persistent data work in triplets of 4KB flash pages. So 50KB of URI data (for a 200-selection wallbox) would inflate to 150KB of flash. With a custom replacement for those built-in APIs, I could probably drop this down a bit, but it would still be more complexity than I wanted to deal with.
So instead, I decided that it made more sense to prepare a folder on the file server just for wallbox-triggered playback. Given that you'd have to organize your music by song codes anyways (to print title strips), it didn't seem like that big of a deal. This way you'd have a single "wallbox" folder with simple and short file names,
By doing this, on-device configuration was mostly automatic. In my own setup, I really only had to configure the base path. For the songs, I just went with the default names and only changed them if my files were something other than MP3s (like M4A or FLAC).
The code that implements this is here: user_webserver.c
The associated HTML and assets are here: html/
Conclusion
Combining everything above with some nasty looking Perl CGI on my home webserver, and fumbling my way through Google Assistant and IFTTT, I eventually ended up with something like this:
This is where everything from Part 3 [Decoding the Pulses] and Part 4 [Inserting Coins] finally comes together into a complete circuit board design. Most of the work here actually took place simultaneously with what I've already described in those posts, so I won't rehash their respective elements. Basically, I did the overall design, then tested and proved out the individual elements on a breadboard, then put it all together into a final schematic and printed circuit board (PCB) layout.
Completing the Schematic
Early Iterations
It all began with a piece of scrap paper and a pencil... Starting with what I had seen on another blog posts about a similar projects, and adding in some of my own ideas about coin switch triggering, a rough schematic started to take shape:
The Pencil Sketch
At this point there were a lot of uncertainties and omissions. I hadn't yet figured out how I was going to filter the signal pulses, wasn't sure which relays I was going to use to toggle the coin switches, and apparently forgot that the relays would actually need a regulated DC voltage to trigger them. Interfacing a microcontroller was also something still far off in the future.
Not long after, I whipped out KiCad and started to layout a real schematic. I selected and tested components for filtering the signal pulses and driving relays to toggle the coin switches. As I proved things out on the breadboard in my lab, I tweaked the schematic until it ended up like this:
Early KiCad Schematic
Most of the components came straight from KiCad's libraries, but a few required some level of customization. KiCad didn't have any diode-protected relay symbols, so I had to make them. KiCad did have a ton of similar-looking rectifier bridge symbols, though, but it took a while to figure out which one best matched what I was using.
Of course, because of the constant changes, the wire routing quickly became a disorganized mess. At this point I decided to pull the various sections apart, and clean up the schematic.
I was also somewhat annoyed that the component for the 74HC14 (Schmitt trigger) didn't expose the power rails in any obvious way. This posed two problems for me: First, unlike many schematics, I had multiple power rails that needed to be distinct and separate. Second, I wanted an obvious place to put a bypass capacitor. Thankfully the newer KiCad libraries (via their Git repos) had redesigned the schematic symbols for this part, and now include a separate "unit" just for connecting power. So I kludged that library into my project, and forged ahead.
The microcontroller was still a bit of an unknown on the design, but I knew I was likely to need an isolated 3.3V supply for it. After some searching, I went ahead and threw in a DC-DC converter (NCS1S2403SC). It also required a custom symbol, albeit a rather easy one to create. The part was a tad pricey, and probably overkill, but it would do the job.
Design More Complete
(At the time I captured this iteration of the schematic, I had obviously forgotten to place a rectifier or any capacitors on the input of the DC-DC converter. I did remember to add those in later.)
Selecting a Microcontroller
Up until this point, I wasn't really sure what kind of microcontroller I wanted to use for this project. Other people have used Raspberry Pi boards, but I was fairly sure I didn't want to do that if it could be avoided. I really wanted something I could more tightly integrate, and that would be more purpose-built. Having a whole Linux system, complete with OS maintenance and startup/shutdown concerns seemed like overkill.
After watching a few too many EEVblog videos about a Nixie Tube Display Project, I became aware of this wonderful little Wi-Fi enabled microcontroller module known as the ESP8266. These modules are typically packaged onto surface mount carrier modules (which package the antenna, memory, and FCC certifications, etc). In Dave's project, he used a WEMOS D1 mini carrier board. This board packaged an ESP-12S module with a few necessary support components (USB port, reset button, etc) and a place to put pin headers.
Since I was unable to directly order that specific module when I was at this stage of design, I went ahead and instead ordered an Adafruit Feather HUZZAH. Its slightly larger, and has a few more support components I didn't really need, but is essentially the same thing. My intent was to use this module as a breadboard-friendly option for development and testing of the software, which I'll go into detail on in a future blog post.
For the actual board, however, I decided to just use an ESP-12S module directly.
Finalizing Revision A
The final step in getting the design ready was to include the ESP-12S into my schematic. Thankfully I found an existing KiCad part library that included the ESP-12E, which I could trivially edit into the part I needed. For the rest of it, I just looked at a variety of "getting started with the ESP8266" blog posts and various schematics for existing carrier boards. All I really needed was a few pull-up resistors, a reset button, a jumper for putting the thing into the flash download startup mode, and a pin header for connecting to the UART.
I didn't bother including a USB controller directly on my board, since it wasn't something I was going to need at runtime. Instead, I opted to just buy an external CP2102 adapter that I could plug in whenever I wanted to download updated firmware.
Eventually I ended up with this schematic:
Ready For Layout
Laying out the PCB
Finding all the Footprints
The first step towards laying out a PCB is making sure you have accurate footprints assigned to all of your components. Depending on what you're building, this is often a mixture of trivial and frustrating.
For the resistors, I knew I was going to use standard-sized "jellybean" 1/4 watt parts from my collection, so I selected reasonable footprints that made sense. (Of course KiCad calls this footprint "R_Axial_DIN0207_L6.3mm_D2.5mm_P10.16mm_Horizontal", so it did take a little bit of work to decide that was the best choice from the library.)
For the capacitors, it was a little bit more of a challenge. I had a collection of usable parts for many of them, but not necessarily of known dimensions. Just to play it safe here, I decided to source all of my capacitors from fresh Digi-Key orders using quality brands. That way I would actually know the specific dimensions of the footprints for each one.
The rectifier bridge used an unusual package type (WOG) that wasn't in the KiCad library. I managed to track down someone's library that contained the right footprint. Making sure the pins all matched the schematic symbol still took some effort, as this part doesn't actually have numbered pins.
The relays were the first fully "custom" footprint that I had to create myself. Fortunately, they were just trivial modification of a common package. I took a DIP-14, deleted the 3 middle pins on either side, and made sure the pin numbering matched up with my schematic symbol.
The DC-DC converter was also a custom footprint, and one that required a bit more work. I found an existing footprint for a similar component from the same manufacturer, and tweaked its footprint based on the datasheet for my component.
Thankfully the component library I based my ESP-12S off of also contained a footprint. I just had to make a few minor modifications to remove certain pins, and I was all set.
Getting it all routed
Now that I had all of my component footprint data sorted out, it was time to actually layout and route the printed circuit board (PCB). I soon discovered that KiCad's PCB designer is actually pretty bare in its choices for things like trace sizes, drill sizes, via sizes, design rules, and the like. Thankfully, I found a few excellent resources that helped me configure it for this project:
PCB Design Tutorial put together by Dave Jones (likely before he started EEVblog)
KiCad Design Rules from OSH Park (who I was planning to use for fabrication)
After spending an afternoon laying out all of my components, and routing all the traces between them, I finally had a PCB design!
PCB Layout (Revision A)
KiCad's 3D view was also pretty cool, and helped with keeping the actual look of the board in mind. I didn't bother creating or fixing any missing/incorrect 3D models, but its still worth showing off:
PCB 3D View
Fabrication Time!
Sending off to OSH Park
I had decided to use OSH Park for making my PCB, mostly because of how easy they make the whole process. I basically just had to upload my ".kicad_pcb" file on their website, and they did the rest.
OSH Park PCB Preview
Finding Design Issues
While I was waiting for the PCB to come back from OSH Park, I decided to spend some more time testing and analyzing my design. During this process, a few initial issues jumped out at me:
The Zener diode I was considering as a voltage clamp on the output of the signal rectifier was a bad idea. It was based on a misunderstanding of a suggestion I saw on a website, and didn't actually work for my use case. This is the one element of the schematic I did not test before doing the PCB layout, and I really should have.
The relays were connecting the coin switches to the Common terminal, rather than to 25VAC. This mistake can apparently be traced all the way back through every one of my schematics, even though my bench test breadboard was doing it right.
The ESP-12S could use another button so I'd have a way of causing my software to start in a "setup" mode. Without doing this, reflashing would be the only way to recover from a broken Wi-Fi configuration.
The first two issues thankfully could be "patched" as part of the board assembly process. The third issue was something I could live with, but would address in a future revision.
Building the Board
After an uncomfortably long (but not really that long) wait, I got the manufactured board back from OSH Park:
Actual PCB (Revision A)
While assembling the board, I took copious notes. There were many things I wanted to verify due to upfront uncertainty, and many other things I discovered during the assembly process. If I was going to do another revision, I wanted to make sure I got all the kinks out.
Fully Assembled PCB (Revision A)
Issues discovered during assembly:
Diode bridge (WOG) footprints had holes and pads that were almost too small. (I managed to make the parts fit, but it was extremely tight.)
DC-DC converter's silkscreen was all wrong. (I expected this when I first figured out the footprint. Thankfully this one was just cosmetic.)
DC-DC converter probably should have filter capacitors on its output. (Just in case.)
Opto-isolator pins were a bit too wide, and took extra bending to fit. (It turned out I had ordered "DIP-4M" parts, whereas I used "DIP-4" footprints.)
UART pin header (J3) had the +V and Gnd silkscreen labels backwards (Big problem if I hadn't caught it, trivial to work around now that I know about it.
The board had tons of dead space, and was way larger than it needed to be.
After assembly, once I got the board working, I made another discovery. Apparently, my ESP-12S (ordered from Electrodragon) had only 1MB of flash. I had been led to believe that the ESP-12S comes with 4MB of flash, so this was quite annoying. Just to be safe, I decided that any future ESP-12S parts would come from another supplier that actually said 4MB on their product page (e.g. Adafruit).
Revision B
Okay, I'll admit that a second PCB revision is kinda silly for a hobby project like this. However, I was taking the design process for this project far more seriously than your average one-off hobby build. So what the heck...
Small Schematic Changes
I fed all the design changes from the previous sections back into the schematic, and arrived at a more "final" revision B:
Schematic (Revision B)
Compacting the Layout
I then spent some more time moving components around, and got the layout quite a bit more compact:
PCB Layout (Revision B)
Running the numbers, this turned out to be more than 8.5 square inches smaller than the Revision A layout!
Back to OSH Park
This time it was simply a matter of uploading and ordering. Since I has gotten the kinks out, I also felt confident enough to actually share the design: OSH Park ~ wallbox.kicad_pcb
OSH Park PCB Preview (Revision B)
Building the Board
This time assembly was very straightforward. All the issues I had discovered in the last revision were resolved, so it was simply a matter of soldering in components.
Assembled PCB (Revision B)
Well, it was mostly simple. At least until I got to the ESP-12S module itself. This time I had the bright idea to attempt "proper" SMD soldering with solder paste. Everything seemed to go fine until I was actually ready to switch the power on... And it didn't work...
During the painful troubleshooting process, I discovered that all the pins on one side of the module were apparently shorted to ground. I tried reheating them with my soldering iron, using solder wick to clean things up, and even using a desoldering gun to suck away excess solder. Nothing seemed to fix the shorts.
I then had this crazy idea to point my ridiculously oversized hot air gun at the thing, in an attempt to reflow the solder. For a long time, nothing seemed to be happening. Then, out of nowhere, the whole ESP-12S module popped right off the PCB!
At this point I tested both the PCB and the module, and couldn't find any evidence of shorts. So I resoldered it the "old fashioned way," and it started working. Phew!
Since the ESP-12S module is not a "real" SMD part, but rather a surface-mountable carrier PCB, I suspect the PCB "sandwich" prevented the solder paste from correctly reflowing around some of the pins.
Bill-Of-Materials
One thing I made sure to do throughout this entire process, was to keep a detailed and accurate bill-of-materials (BOM) for the project. Initially I did this with a plain text file, but eventually I did the grunt work to move all the data into the KiCad schematic itself. The BOM included part descriptions, datasheet links, part numbers, and vendor information. This made it easy to use various companion tools to generate HTML or CSV versions for easy sharing and part ordering, and provided a handy reference during assembly and testing.
