UPDATE [2018] - This was written up a long time ago. Things have evolved. Myself included. I often wondered why I didn’t mention why I chose not to use ROS when I wrote this. My latest evolution of this idea was started in 2017 and is named R7.

---

Basis for experimentation around interfacing a computer to the physical world.

This post describes the current state a software framework I have been working on randomly over the past year. It was designed to allow for advanced experimentation around concepts that come in to play when interfacing a computer to the physical world.

The implementation is far enough along that I’ve tested end-to-end functionality through all the layers described below. The joystick controlled RGB LED project I wrote about previously is using this framework with all of the specific technologies and tools that I have baselined here.

Layers

Purpose

At first glance this framework might seem over engineered. However, I have thought a lot about the layers along with some of things I hope to accomplish eventually. My constant goal has been to keep things as simple as possible, but no simpler. Some of my ideas involve doing things that aren’t easily possible when using off-the-shelf components. For example, I want to have enough control that I can custom program ease-in/out movements of servos (both RC Servos and Dynamixel Servos ). Most of the servo controllers on the market do not have that capability. In earlier experiments I tried programming these custom movements in the PC layer using an SSC-32 for actual servo control, but the communications latency (RS232, buffers, microcontroller, etc) made the end result unreliable and not as smooth as I had hoped for.

Controller/Keyboard

A controller could be a joystick, a mouse, a Belkin Nostroma n52te , a test computer keyboard, or any combination. The controller layer could also eventually include a head tracking device like the Natural Point TrackIR.

User Interface

The computer provides the GUI and all visuals. In all the other layers of this framework I have decided on specific technology and product choices for a baseline. However, I am still undecided as to which GUI framework/platform I want to use as a baseline. Options for the GUI layer include:

• JavaFX 2.0 (very different from JavaFX 1.0)
• Netbeans Platform (Swing. Not same thing as the Netbeans IDE.)
• HTML/JavaScript with NodeWebkit.js (or possibly wrapped in QtWebEngine Window )
• Qt Framework
• Flash/Flex/AIR
• Eclipse RCP

I keep holding off because I only have so much time to learn new things. The state of GUI development for business is a bit up-in-the-air presently.

Keeping the GUI as its own self-contained layer means it’s possible to make use of the remaining parts of this framework without any GUI at all. That’s what I’ve been doing mostly to-date because GUI’s take so much time to develop and generally require a clear goal.

Most recently I’ve been experimenting by creating a Java wrapper with an embedded Groovy engine. This lets me prototype console applications quickly using a dynamic language, but with all the power of the JDK and many available Java libraries.

High Level Logic on PC

Hobbyists, and even many professionals, seem to want to do as much as possible on the on-board computers or microcontrollers. I am guessing many of them come from an EE background. I am of the opposite mind though. I want to do as much as possible on the PC. My reasons for preferring to develop on PC vs. microcontroller:

• More advanced development tools available
• Often using same tools I already use for business app development
• More resources available (CPU and Memory)
• More language/platform options &
• Easier to test and debug.

The only reason not to do something on the PC is that it would be too slow. As it turns out in robotics, that is often the case, and so highly optimized routines and lower-level microcontroller programming is required. Still, being able to do as much of the high-level, non real-time, logic on the PC as possible opens up a lot of possibilities.

TCP/IP Server and Serial Port Proxy

The TCP/IP server is a custom C++ server that listens on a generic network socket for clients. Its primary purpose is to convert network messages to serial messages that get sent to the microcontroller and vice-a-versa.

The C++ server was written in cross-platform C++, making significant use of Boost . Boost provides the cross-platform threading, network socket abstraction, and serial port abstraction. (The network and serial port programming share the same paradigm in the Boost::ASIO library ). Everything in Boost is impressive, but the ASIO library especially strikes me as being written by the elite of C++ craftsmen.) If you want to get a feel for the type of programming involved, you can see snippets of my code in a question I posted to StackOverflow while working on this a while ago: How should I delete a child from parent’s slot? .

The C++ server is very generic in its base from. As part of the initial setup protocol when a client connects (all of which is custom), the client instructs the server which serial port to open and provides port configuration parameters. The server is meant to be an executable piece of code requiring little to no configuration that can be run almost anywhere to provide network to serial conversion. The ability to run it almost anywhere is important because it means that the server can be run on the PC workstation, or it can be run on an on-board computer (perhaps even one as small as a Linux Gumstix .) C/C++ is not the easiest of languages, but it can be one of the most cross-platform in source code form. After developing the C++ server, I came across a few open source projects that are essentially doing the same thing. (i.e. TCP/IP to Serial proxying). However, I didn’t see anything open source that could be easily made cross-platform or that looked as robust and as flexible as the server I developed myself using ASIO.

Instead of C++, I could have used Java. I have used RXTX for serial port access and Bluecove for Bluetooth access in Java successfully. However, there were minor issues with both. (RXTX wasn’t super reliable. Bluecove experienced random delays when connecting.) More importantly though, using Java would have severely limited options for running the serial port proxy on embedded devices.

Native Capabilities

Another benefit to using a custom C++ server is that it provides the option to write very low-level routines if needed. Most of the better GUI frameworks and platforms I’m familiar with (Java, Flex/Flash/AIR, Java) have limited capabilities for interacting with low-level OS functions and hardware directly. (Java provides JNI/JNA , but that would mean yet another conversion layer.) There are ways of integrating native C/C++ code with these platforms, but the techniques and/or libraries are specific to each platform. By providing a smartly designed network interface, the native interfacing layer is common and sharable with all the front-end platforms.

If needed, I can implement C/C++ routines when and where needed, which can then be easily called by any client via the uniform network interface. The network latency might be a problem for certain types of problems, but this is still a great addition to the framework’s capabilities.

Microcontroller

The microcontroller is where the programming hits the metal. This type of programming is often referred to as embedded programming or realtime programming.

There are significant differences in the various types of microcontrollers and related programming distinctions, but these differences would seem arcane to anyone outside of the embedded space. Keep in mind that a company might be producing hundreds of thousands of devices with an embedded microcontroller and being able to select a microcontroller that costs even a few cents less at bulk quantities can be a significant cost difference. For this reason many microcontrollers are highly specialized and limited in their capabilities. Luckily, hobbyists don’t have to make the same trade-offs and the sorts-of microcontrollers I’m interested in tend to be more powerful and feature rich.

One of the more popular microcontroller platforms for hobbyists currently is the Aurdino . However, I wanted something more powerful, so my baseline is a Parallax Propeller microcontroller. ( Parallax is probably best known for creating the BasicStamp line of microcontrollers.) The propellor is available in various forms. I started with the education kit (breadboard and 40-pin DIP) so I could learn the Propellor’s Spin language. I then purchased a PropStick USB:

This is a smaller version of the chip with supporting voltage regulators and USB interface already soldered on. I put this onto a custom board where I soldered headers for connecting servos. The end result is quite powerful (for a microcontroller), easy to program (via USB), and yet is smaller than most dedicated servo controllers.

The only technical difficulty I’ve run in to so far with the propellor that I probably wouldn’t have with the Arduino, is that the Propellor is 3.3v, whereas the Arduino is 5.0v. Because of that I had to be careful with the power rails to the servos. (I think I need to redo my current power connection setup regardless. I didn’t count on how just how much amperage could be needed if all servos are active at the same time.)

A key feature of the Propellor design is the notion of "Cogs" …a paradigm for multithreaded like programming:

The Propellor design does not use interrupts which is a big difference as compared to many other microcontrollers. The cog design is similar to having 8 CPUs each being given a slice of (real)time to operate with relatively safe access to shared resources. I have used this capability to simplify the programming such that one cog handles communications to/from the serial connection, two more cogs are used to track and drive PWM signals to the connected servos.

On the device there could be other components and addons, such as a dedicated servo controller like the Lynxmotion SSC-32 , sensors, motor drivers, and other hardware interfacing components. The expectation as that my custom microcontroller brokers all of these devices to the higher level layers of the framework. Common protocols and related terms at this level include PWM, I2C, RS232, RS485, TTL, SPI, 2-wire, Relays, H-Bridges, MOSFET, ESCs, BECs, etc.