Monday, November 20, 2017

Building a Seeburg Wall-O-Matic Interface (Part 6)

Developing the Software

This post is definitely not going to completely describe the software. The software is always going to be a work-in-progress, so follow the Git repository for more specific updates. Instead, its going to describe a bit about the development setup, and some basics about what the software is doing.

Wall-O-Matic Microcontroller Interface ESP8266 Code [github.com]

Preparing a development rig

The first step in this stage of the project was to actually have a development board to work with. While I could have simply used my real circuit board (as described in the previous post), doing so had a number of disadvantages:
  • Somewhat tedious to switch in-and-out of flash download mode
  • Required an extra adapter to connect to USB
  • Designed to be powered by a bulky transformer connected to AC mains
  • No way to test the coin switch outputs without a multimeter
  • No way to test the pulse decoding input without a real wallbox
In other words, using the real circuit board was not a solution well suited to an office desk environment that was outside of my electronics workbench.

To do the majority of the software development, I really only needed to test the ESP-12S itself and basic interactions on its I/O pins. Thankfully, this did not actually require the full circuit board. So, to make my life easier, I cobbled together a little breadboard development rig.

Development Rig Breadboard

For the ESP-12S part of this rig, I used an Adafruit Feather HUZZAH. Its a convenient breakout board that includes a USB port, reset button, and some nifty wiring that allows the USB serial controller to automatically switch the ESP-12S in and out of flash download startup mode.

To simulate the signal pulses of the wallbox, I used an Arduino Nano. This module also has its own USB port, and I/O pins that could easily be programmed to simulate any digital pulse stream I needed. Since I'm not testing the full filtering path, I really only need to simulate a clean series of logic-level pulses. The code that runs on this Arduino can be found here: [PulseSim.ino]

For the coin switches, three LEDs in different colors were enough. After all, I just needed to test that I could make them blink.

Since the Arduino is a 5V system, and the ESP8266 is a 3.3V system, I leveraged my existing bin of components to link these together. I used a spare opto-isolator and inverter, which did the trick. This had the benefit of making the two microcontroller modules completely isolated from each other, so they could be powered independently.

In schematic form, it looked like this:
Development Rig Schematic

Setting up the toolchain

The easiest way to get the development tools up and running was to leverage the "esp-open-sdk" project.  This project is essentially a Makefile and some scripts that downloads, compiles, and configures everything you need to write code for the ESP8266. This part was surprisingly painless, and decently covered in that project's README.

Decoding Pulses (For Real)


When I was originally figuring out the pulse protocol and filtering circuitry, I used an Arduino-based test decoder. The code for this basically used the "pulseIn()" function in a loop. Upon further investigation, I found that "pulseIn()" just busy-waits on the I/O pin. While perfectly fine for basic testing, this was definitely not a good solution for a complete system.

The approach I took here involved configuring an interrupt to trigger on any transition on the signal pulse's input pin. Inside the interrupt handler function, I then populated an array with the pulse durations and pulse gap widths and kicked off a timer.  If no pulses happened for a certain amount of time, the timer callback function would then inspect the array and determine what the song selection was. This approach ended up working quite well.

The code that implements this is here: user_wb_selection.c

Inserting Coins (For Real)

While precise timing is really not necessary for the wallbox's coin switch mechanism, I still went and measured how long a real coin would actually trip the switches for (40-60ms). I then wrote some simple timer functions that would trigger the selected coin switch for the appropriate amount of time.

The code that implements this is here: user_wb_credit.c

Interfacing with Sonos

This is the step where things started to get complicated. Some of the blog posts I read at the outset of this project had led me to believe it was just a matter of spitting the right HTTP POST at a Sonos speaker's IP address, but is anything actually that simple?  Especially if you want a robust and reliable solution?

As I worked through this process, I had decided that there were certain things I definitely wanted to have:
  • Sonos device configuration should be easy (i.e. no hard-coded IP addresses)
  • Can't make any assumptions as to the state of the Sonos speaker
  • The Sonos device's playlist should be treated as if it was a jukebox
The goal was not only to make setup easy, but to also ensure that I should never have to fiddle the Sonos app on my phone to get things into (or back into) a functional state. The Wall-O-Matic interface should function as standalone as technically possible.
Of course doing all of this meant that I needed to figure out quite a bit more of the Sonos protocol than a single enqueue request.

To figure all this out, I spent a lot of time staring at Wireshark while fiddling with both the official Sonos app and the node-sonos-http-api project's utility commands.

I'm not going to even attempt to document the Sonos protocols here, as that would take way too much time and effort to write up. I'll just be giving a short summary of the parts I used, then linking to my code that implements them.

Discovery Protocol

The Sonos discovery protocol is based on SSDP, and involves two kinds of device discovery: explicit searches and notifications.

Explicit searches begin by sending a UDP broadcast to IP address 239.255.255.250 (port 1900) with the following payload:

M-SEARCH * HTTP/1.1
HOST: 239.255.255.250:reservedSSDPport
MAN: ssdp:discover
MX: 1
ST: urn:schemas-upnp-org:device:ZonePlayer:1

All Sonos devices on the network will then reply to the sender, by sending something like this to the source port of the discovery request:

HTTP/1.1 200 OK
CACHE-CONTROL: max-age = 1800
EXT:
LOCATION: http://192.168.1.42:1400/xml/device_description.xml
SERVER: Linux UPnP/1.0 Sonos/37.12-45270 (ZP120)
ST: urn:schemas-upnp-org:device:ZonePlayer:1
USN: uuid:RINCON_D6E477CEC684A1400::urn-schemas-upnp-org:device:ZonePlayer:1
. . .

Additionally, Sonos devices will periodically broadcast their presence on port 1900 without any discovery requests. When that happens, the payload looks similar:

NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age = 1800
LOCATION: http://192.168.1.42:1400/xml/device_description.xml
NT: upnp:rootdevice
NTS: ssdp:alive
SERVER: Linux UPnP/1.0 Sonos/37.12-45270 (ZP120)
USN: uuid:RINCON_D6E477CEC684A1400::upnp:rootdevice
. . .

Regardless of how it came in, I collected the IP address, port, and UUID from these payloads. Then, to get the zone name itself, I made an HTTP GET to the following URL:

http://192.168.1.42:1400/status/zp

The response to this was a big XML blob that contained the zone name, as well as other device information.

This info was then collected in a data structure, and later used as follows:
  • IP and Port - Used to actually connect to the device and to send it commands
  • UUID - Used as the reference to the device kept in the software configuration, and used as part of certain control requests. 
  • Zone Name - Used for display purposes, so the UI could give friendly information on which zone was selected.
The code that implements this is here: user_sonos_discovery.c

Control Requests

To instruct the Sonos device to actually do things (e.g. add song to queue, play, query status, etc), I had to implement the following control requests:
  • AddURIToQueue - Adds a file or stream URI to the playback queue
  • GetPositionInfo - Gets position info on the current track
  • SetAVTransportURI - Sets the transport URI to play (e.g. stream source or local file queue)
  • Seek - Select the queued track to play
  • Play - Start or resume playback
The code that implements this is here: user_sonos_request.c

Event Subscriptions

Unfortunately, the normal control interface provides no way to distinguish between the playing and paused states. The only way to do this, was to listen for event notifications. There's another protocol Sonos devices support where you can tell them to notify you whenever there's a state change. Thankfully, these notifications include the play/paused state.

The code that implements this is here: user_sonos_listener.c

Putting it all together

At boot, the device discovery listener is started and a discovery request is broadcast. As soon as the configured Sonos device UUID is detected, it is set as the active device and an event subscription request is made.

When a song selection is made on the wallbox, the following sequence of operations are initiated:
  • Construct a track URI for the selected song
  • Add the track URI to the Sonos device queue (AddURIToQueue)
  • Get the current position info (GetPositionInfo)
  • If the current transport is not a file URI, then switch to the local queue (SetAVTransportURI)
  • If the current transport is a file URI, and currently playing, then do nothing
  • Otherwise:
    • If not currently playing, start playing
    • If on a previous track, and not currently playing, then seek to the added track and start playing
    • If on a previous track, and paused, then seek to the next (or added) track and start playing.

The code that implements this is here: user_sonos_client.c

Providing a User Interface

At runtime, the wallbox itself kinda is the user interface for the project. Well, with one exception: inserting coins. I needed some way to instruct my board to "insert a nickel/dime/quarter" remotely, either via a manual interaction (i.e. click a button on an app/website) or via some sort of home automation setup. To accomplish this, I decided I'd simply expose some sort of basic HTTP API for triggering the coin switch relays.

At setup time, it was a different story. There was actually a lot to configure for this thing to work correctly:
  • Wi-Fi Network Setup
  • Selecting the Sonos device to play through
  • Selecting the type of the connected wallbox
  • Configuring which song file to play for each wallbox selection code
Some might simply bake all this into the firmware. I really didn't want to do this, since you'd never design a "real" product like that.

So the first thing I did, was to embed a web server into the device. To do this, I leveraged the libesphttpd project (and its sample esphttpd app). This project provides a simple webserver written against the ESP APIs, and a fair amount of sample/utility code to handle support functions (Wi-Fi setup and firmware updating) that I knew I was going to need.

As I began to explore this project, I soon noticed that it hadn't been updated in a while. However, there seemed to be a number of forks that had been picking up improvements and fixes. Ultimately, I settled on using Chris Morgan's fork of the project.

After several iterations, I ended up with a (mobile friendly) landing page that looked like this:
Local Web Interface

The setup and firmware pages were pretty much straight lifts from the esphttpd sample project, albeit with some minor style tweaks. The Sonos zone selection page was also fairly simple.  The most complex of these pages was the "Wallbox Configuration" page:

Wallbox Configuration Page
The "wallbox type" determines how many song selections there are, how they're numbered, and what the format of the wallbox's signal pulses are. (Contrary to a previous post, I did eventually get the model "200" mostly working and added code to support its signal pulse format.  The format is different from the "100", and actually slightly simpler.)

The (long) "base folder path" and the (short) "song" filenames are combined to form the playback URI for each song selection. Buttons help automate the otherwise-tedious entry process.

The layout of this page reflects the approach I took towards configuring the playback URIs. While I could have allowed a completely custom per-song-selection URI, doing so would have required quite a lot of configuration memory.

The ESP SDK has APIs for "safe" persistent data work in triplets of 4KB flash pages. So 50KB of URI data (for a 200-selection wallbox) would inflate to 150KB of flash.  With a custom replacement for those built-in APIs, I could probably drop this down a bit, but it would still be more complexity than I wanted to deal with.

So instead, I decided that it made more sense to prepare a folder on the file server just for wallbox-triggered playback. Given that you'd have to organize your music by song codes anyways (to print title strips), it didn't seem like that big of a deal. This way you'd have a single "wallbox" folder with simple and short file names,

By doing this, on-device configuration was mostly automatic. In my own setup, I really only had to configure the base path. For the songs, I just went with the default names and only changed them if my files were something other than MP3s (like M4A or FLAC).

The code that implements this is here: user_webserver.c
The associated HTML and assets are here: html/

Conclusion

Combining everything above with some nasty looking Perl CGI on my home webserver, and fumbling my way through Google Assistant and IFTTT, I eventually ended up with something like this:


Monday, November 06, 2017

Building a Seeburg Wall-O-Matic Interface (Part 5)

Designing and Building the Circuit

This is where everything from Part 3 [Decoding the Pulses] and Part 4 [Inserting Coins] finally comes together into a complete circuit board design. Most of the work here actually took place simultaneously with what I've already described in those posts, so I won't rehash their respective elements. Basically, I did the overall design, then tested and proved out the individual elements on a breadboard, then put it all together into a final schematic and printed circuit board (PCB) layout.

Completing the Schematic

Early Iterations

It all began with a piece of scrap paper and a pencil... Starting with what I had seen on another blog posts about a similar projects, and adding in some of my own ideas about coin switch triggering, a rough schematic started to take shape:
The Pencil Sketch
At this point there were a lot of uncertainties and omissions. I hadn't yet figured out how I was going to filter the signal pulses, wasn't sure which relays I was going to use to toggle the coin switches, and apparently forgot that the relays would actually need a regulated DC voltage to trigger them. Interfacing a microcontroller was also something still far off in the future.

Not long after, I whipped out KiCad and started to layout a real schematic. I selected and tested components for filtering the signal pulses and driving relays to toggle the coin switches. As I proved things out on the breadboard in my lab, I tweaked the schematic until it ended up like this:
Early KiCad Schematic
Most of the components came straight from KiCad's libraries, but a few required some level of customization. KiCad didn't have any diode-protected relay symbols, so I had to make them. KiCad did have a ton of similar-looking rectifier bridge symbols, though, but it took a while to figure out which one best matched what I was using.

Of course, because of the constant changes, the wire routing quickly became a disorganized mess. At this point I decided to pull the various sections apart, and clean up the schematic.

I was also somewhat annoyed that the component for the 74HC14 (Schmitt trigger) didn't expose the power rails in any obvious way. This posed two problems for me: First, unlike many schematics, I had multiple power rails that needed to be distinct and separate. Second, I wanted an obvious place to put a bypass capacitor. Thankfully the newer KiCad libraries (via their Git repos) had redesigned the schematic symbols for this part, and now include a separate "unit" just for connecting power. So I kludged that library into my project, and forged ahead.

The microcontroller was still a bit of an unknown on the design, but I knew I was likely to need an isolated 3.3V supply for it. After some searching, I went ahead and threw in a DC-DC converter (NCS1S2403SC). It also required a custom symbol, albeit a rather easy one to create. The part was a tad pricey, and probably overkill, but it would do the job.

Design More Complete
(At the time I captured this iteration of the schematic, I had obviously forgotten to place a rectifier or any capacitors on the input of the DC-DC converter. I did remember to add those in later.)

Selecting a Microcontroller

Up until this point, I wasn't really sure what kind of microcontroller I wanted to use for this project. Other people have used Raspberry Pi boards, but I was fairly sure I didn't want to do that if it could be avoided. I really wanted something I could more tightly integrate, and that would be more purpose-built. Having a whole Linux system, complete with OS maintenance and startup/shutdown concerns seemed like overkill.

After watching a few too many EEVblog videos about a Nixie Tube Display Project, I became aware of this wonderful little Wi-Fi enabled microcontroller module known as the ESP8266. These modules are typically packaged onto surface mount carrier modules (which package the antenna, memory, and FCC certifications, etc). In Dave's project, he used a WEMOS D1 mini carrier board. This board packaged an ESP-12S module with a few necessary support components (USB port, reset button, etc) and a place to put pin headers.

Since I was unable to directly order that specific module when I was at this stage of design, I went ahead and instead ordered an Adafruit Feather HUZZAH. Its slightly larger, and has a few more support components I didn't really need, but is essentially the same thing. My intent was to use this module as a breadboard-friendly option for development and testing of the software, which I'll go into detail on in a future blog post.

For the actual board, however, I decided to just use an ESP-12S module directly.

Finalizing Revision A

The final step in getting the design ready was to include the ESP-12S into my schematic. Thankfully I found an existing KiCad part library that included the ESP-12E, which I could trivially edit into the part I needed. For the rest of it, I just looked at a variety of "getting started with the ESP8266" blog posts and various schematics for existing carrier boards. All I really needed was a few pull-up resistors, a reset button, a jumper for putting the thing into the flash download startup mode, and a pin header for connecting to the UART.

I didn't bother including a USB controller directly on my board, since it wasn't something I was going to need at runtime. Instead, I opted to just buy an external CP2102 adapter that I could plug in whenever I wanted to download updated firmware.

Eventually I ended up with this schematic:
Ready For Layout

Laying out the PCB

Finding all the Footprints

The first step towards laying out a PCB is making sure you have accurate footprints assigned to all of your components. Depending on what you're building, this is often a mixture of trivial and frustrating.

For the resistors, I knew I was going to use standard-sized "jellybean" 1/4 watt parts from my collection, so I selected reasonable footprints that made sense. (Of course KiCad calls this footprint "R_Axial_DIN0207_L6.3mm_D2.5mm_P10.16mm_Horizontal", so it did take a little bit of work to decide that was the best choice from the library.)

For the capacitors, it was a little bit more of a challenge. I had a collection of usable parts for many of them, but not necessarily of known dimensions. Just to play it safe here, I decided to source all of my capacitors from fresh Digi-Key orders using quality brands. That way I would actually know the specific dimensions of the footprints for each one.

The rectifier bridge used an unusual package type (WOG) that wasn't in the KiCad library. I managed to track down someone's library that contained the right footprint. Making sure the pins all matched the schematic symbol still took some effort, as this part doesn't actually have numbered pins.

The relays were the first fully "custom" footprint that I had to create myself. Fortunately, they were just trivial modification of a common package. I took a DIP-14, deleted the 3 middle pins on either side, and made sure the pin numbering matched up with my schematic symbol.

The DC-DC converter was also a custom footprint, and one that required a bit more work. I found an existing footprint for a similar component from the same manufacturer, and tweaked its footprint based on the datasheet for my component.

Thankfully the component library I based my ESP-12S off of also contained a footprint. I just had to make a few minor modifications to remove certain pins, and I was all set.

Getting it all routed

Now that I had all of my component footprint data sorted out, it was time to actually layout and route the printed circuit board (PCB). I soon discovered that KiCad's PCB designer is actually pretty bare in its choices for things like trace sizes, drill sizes, via sizes, design rules, and the like. Thankfully, I found a few excellent resources that helped me configure it for this project:
After spending an afternoon laying out all of my components, and routing all the traces between them, I finally had a PCB design!
PCB Layout (Revision A)

KiCad's 3D view was also pretty cool, and helped with keeping the actual look of the board in mind. I didn't bother creating or fixing any missing/incorrect 3D models, but its still worth showing off:
PCB 3D View

Fabrication Time!

Sending off to OSH Park

I had decided to use OSH Park for making my PCB, mostly because of how easy they make the whole process. I basically just had to upload my ".kicad_pcb" file on their website, and they did the rest.

OSH Park PCB Preview

Finding Design Issues

While I was waiting for the PCB to come back from OSH Park, I decided to spend some more time testing and analyzing my design. During this process, a few initial issues jumped out at me:
  • The Zener diode I was considering as a voltage clamp on the output of the signal rectifier was a bad idea. It was based on a misunderstanding of a suggestion I saw on a website, and didn't actually work for my use case. This is the one element of the schematic I did not test before doing the PCB layout, and I really should have.
  • The relays were connecting the coin switches to the Common terminal, rather than to 25VAC. This mistake can apparently be traced all the way back through every one of my schematics, even though my bench test breadboard was doing it right.
  • The ESP-12S could use another button so I'd have a way of causing my software to start in a "setup" mode. Without doing this, reflashing would be the only way to recover from a broken Wi-Fi configuration.
The first two issues thankfully could be "patched" as part of the board assembly process. The third issue was something I could live with, but would address in a future revision.

Building the Board

After an uncomfortably long (but not really that long) wait, I got the manufactured board back from OSH Park:
Actual PCB (Revision A)
While assembling the board, I took copious notes. There were many things I wanted to verify due to upfront uncertainty, and many other things I discovered during the assembly process. If I was going to do another revision, I wanted to make sure I got all the kinks out.

Fully Assembled PCB (Revision A)


Issues discovered during assembly:
  • Diode bridge (WOG) footprints had holes and pads that were almost too small. (I managed to make the parts fit, but it was extremely tight.)
  • DC-DC converter's silkscreen was all wrong. (I expected this when I first figured out the footprint. Thankfully this one was just cosmetic.)
  • DC-DC converter probably should have filter capacitors on its output. (Just in case.)
  • Opto-isolator pins were a bit too wide, and took extra bending to fit. (It turned out I had ordered "DIP-4M" parts, whereas I used "DIP-4" footprints.)
  • UART pin header (J3) had the +V and Gnd silkscreen labels backwards (Big problem if I hadn't caught it, trivial to work around now that I know about it.
  • The board had tons of dead space, and was way larger than it needed to be.
After assembly, once I got the board working, I made another discovery. Apparently, my ESP-12S (ordered from Electrodragon) had only 1MB of flash. I had been led to believe that the ESP-12S comes with 4MB of flash, so this was quite annoying. Just to be safe, I decided that any future ESP-12S parts would come from another supplier that actually said 4MB on their product page (e.g. Adafruit).

Revision B

Okay, I'll admit that a second PCB revision is kinda silly for a hobby project like this. However, I was taking the design process for this project far more seriously than your average one-off hobby build. So what the heck...

Small Schematic Changes

I fed all the design changes from the previous sections back into the schematic, and arrived at a more "final" revision B:
Schematic (Revision B)

Compacting the Layout

I then spent some more time moving components around, and got the layout quite a bit more compact:
PCB Layout (Revision B)

Running the numbers, this turned out to be more than 8.5 square inches smaller than the Revision A layout!

Back to OSH Park

This time it was simply a matter of uploading and ordering. Since I has gotten the kinks out, I also felt confident enough to actually share the design:
OSH Park ~ wallbox.kicad_pcb

OSH Park PCB Preview (Revision B)

Building the Board

This time assembly was very straightforward. All the issues I had discovered in the last revision were resolved, so it was simply a matter of soldering in components.
Assembled PCB (Revision B)
Well, it was mostly simple. At least until I got to the ESP-12S module itself. This time I had the bright idea to attempt "proper" SMD soldering with solder paste. Everything seemed to go fine until I was actually ready to switch the power on... And it didn't work...

During the painful troubleshooting process, I discovered that all the pins on one side of the module were apparently shorted to ground. I tried reheating them with my soldering iron, using solder wick to clean things up, and even using a desoldering gun to suck away excess solder. Nothing seemed to fix the shorts.

I then had this crazy idea to point my ridiculously oversized hot air gun at the thing, in an attempt to reflow the solder. For a long time, nothing seemed to be happening. Then, out of nowhere, the whole ESP-12S module popped right off the PCB!

At this point I tested both the PCB and the module, and couldn't find any evidence of shorts. So I resoldered it the "old fashioned way," and it started working. Phew!

Since the ESP-12S module is not a "real" SMD part, but rather a surface-mountable carrier PCB, I suspect the PCB "sandwich" prevented the solder paste from correctly reflowing around some of the pins.

Bill-Of-Materials

One thing I made sure to do throughout this entire process, was to keep a detailed and accurate bill-of-materials (BOM) for the project. Initially I did this with a plain text file, but eventually I did the grunt work to move all the data into the KiCad schematic itself. The BOM included part descriptions, datasheet links, part numbers, and vendor information. This made it easy to use various companion tools to generate HTML or CSV versions for easy sharing and part ordering, and provided a handy reference during assembly and testing.

Bill-Of-Materials Snippet

Project Resources