Update: You can follow the discussion of this article around the web on HN, Adafruit, Hackaday, and Hackster.io!

This article explores the research and development journey behind my new sensor system, OptiGap, a key component of my PhD research. I’m writing this in a storytelling format to offer insights into my decision-making process and the evolution leading to the final implementation. It should hopefully provide a glimpse into the sometimes-shrouded world of PhD research and may appeal to those curious about the process. For a deeper dive into technical specifics, simulations, and existing research on this subject, my dissertation is available online here.

What does it do?

In very general terms, this sensor is basically a rope that if bent can tell you where along its length you bent it. The fancy term for that is “bend localization.”

OptiGap’s application is mainly within the realm of soft robotics, which typically involves compliant (or ‘squishy’) systems, where the use of traditional sensors is often not practical. The name OptiGap, a fusion of “optical” and “gap,” reflects its core principle of utilizing air gaps within flexible optical light pipes to generate coded patterns essential for bend localization.

How the OptiGap Sensor System Started

The idea for OptiGap came about while I was experimenting with light transmission through various light pipes (optical cables) for use as a bend detection sensor. I was initially trying to see how I could effectively “slow down” light through the fiber…a seemingly straightforward task, right?

During this process, I attached a section of clear 3D printer filament (1.75mm TPU) to a piece of tape measure for an experiment and incidentally discovered that when I bent the tape measure (and filament) at the spot where the electrical tape was attached, there was a significant drop in light transmission. I hypothesized that this was because the sticky residue of the electrical tape was causing the filament to stretch, which in turn reduced the light transmission.

To verify this hypothesis, I attached a longer piece of TPU to a tape measure and began bending it at various points to observe how light transmission would change.

I wrote a small Linux I2C driver for the VL53L0X ToF sensor to run on a Raspberry Pi and push the data to a socket using ZeroMQ. I then created a rough GUI in Python to pull the sensor data from the socket and visualize the light transmission data in realtime, shown in the GIF below, which very quickly validated my hypothesis. This validation marked the “Eureka!” moment that sparked the eventual development of the OptiGap sensor.

The OptiGap Realization

I realized that since I could control where the light was being attenuated, I could use this to encode information about the position of the bend on the sensor. Using electrical tape was not a practical solution, so I started looking for a more reliable and consistent way to create these attenuations. This led me to the idea of cutting the filament and then reattaching it together using a flexible rubber (silicone) sleeve, leaving a small air gap, as shown in the image below.

The main working principle of the air gap is that translation and/or rotation of one light pipe face relative to the other changes the fraction of light transmitted across the gap. The greater the bend angle, the more light escapes across the gap. The resulting change in intensity of the optical signal can then be correlated with known patterns for use as a sensor.

The Big Idea

I then proceeded to test this idea by creating multiple air gaps in a row and bending the filament to measure the attenuation.

As depicted in the GIF below, the optical intensity decreases at each air gap, with a more noticeable decrease as the bend angle increases. This initial experimentation served as proof of concept, demonstrating the feasibility of the idea. It led to the formulation of my final hypothesis of utilizing a pattern of these air gaps to encode information regarding the sensor’s bending and employing a naive Bayes classifier on a microcontroller to decode the bend location.

This concept resembles the functionality of a linear encoder. Linear encoders gauge an object’s linear movement, typically comprising a slider rail with a coded scale akin to a measuring ruler and a sensing head that moves across this scale to read it. Linear (absolute) encoders emit a distinct code at each position, ensuring consistent identification of displacement.

The OptiGap system, functioning like an absolute encoder, would encode absolute positions using patterns of bend-sensitive air gaps along parallel light pipes, effectively serving as a singular fiber optic sensor.

Encoding the Bend Location using Inverse Gray Code

Absolute encoders commonly employ Gray code, a binary system where two successive values differ in only one bit. This property allows for various applications, including error checking. However, Gray code isn’t optimal for the OptiGap sensor system. Here, we aim for consecutive values to differ by the maximum number of bits to facilitate easier differentiation. This necessity gave rise to Inverse Gray code.

Inverse Gray code is a binary code where two successive values differ by the maximum (n-1) number of bits. To implement this, I simply create cuts in the filament wherever there’s a “1” in the Inverse Gray code sequence. This approach can scale to any bit number. For the prototype, I utilized 3 bits, providing 8 possible positions.

Visualization of the OptiGap Sensor System

The illustration below depicts the signal patterns of the OptiGap sensor system for each bend position using three fibers. By employing a naive Bayes classifier, the sensor system can discern bend positions based on signal patterns. The third graph represents actual sensor data from the prototype system, utilized for training the classifier on the microcontroller.

The OptiGap Prototype

I proceeded to construct a prototype of the OptiGap sensor system, utilizing 3 strands of clear TPU 3D printer filament, each featuring a distinct pattern of air gaps. The image below showcases the filament just before cutting, with the cut pattern indicated on a piece of tape.

For the prototype, I employed a commercial 3:1 fiber optic coupler to merge the light from the 3 strands into a single fiber optic cable, resulting in the completion of the sensor prototype, as depicted below.

This marked the final phase of validating the hypothesis and operational theory behind the OptiGap sensor.

Reducing the Physical Size

The initial prototype proved to be large and bulky, primarily due to the size of the 3D printer filament used. Drawing from previous experience, I recognized that PMMA (plastic) optical fiber offered a smaller and more flexible alternative suitable for this application. Consequently, I assessed 500, 750, and 1000 micron unjacketed PMMA optical fibers from Industrial Fiber Optics, Inc. for the sensor strands, resulting in a significant reduction in sensor size.

I conducted tests on all three types of fibers to evaluate their light transmission and flexibility. Among them, the 500 micron fiber emerged as the optimal choice overall, although all three exhibited sufficient flexibility for this application.

Reducing the Optical Transceiver Complexity

I decided to switch from using the complex VL53L0X ToF sensor to a simple photodiode and IR LED setup to reduce the complexity of the system and to increase modularity. This also allowed me to use a  microcontroller to read the sensor data, which was a significant improvement over the initial prototype.

I then created a demo system for the sensor based around an STM32 microcontroller and a photodiode/IR LED setup.

Realtime Machine Learning on a Microcontroller

The final stage in developing the OptiGap sensor system involved integrating a naive Bayes classifier onto the STM32 microcontroller to decode the bend location from the sensor data. I opted for a naive Bayes classifier due to its efficiency compared to if-statements or lookup tables, its capability to handle new or previously unseen data, and its potential for increased accuracy by considering relationships between multiple input variables.

Implementing the naive Bayes classifier proved to be relatively straightforward. This classifier is a probabilistic model based on applying Bayes’ theorem to determine how a measurement can be assigned to a particular class, with the class representing the bend location in this context. I utilized the Arm CMSIS-DSP library for the classifier implementation.

Fitting the Sensor Data

The initial step in integrating the classifier was to fit the sensor data to a Gaussian distribution for each air gap pattern. To expedite this process, I developed a Python GUI for rapid labeling and fitting of the data using GNB (Gaussian Naive Bayes) from the scikit-learn library.

I later improved this UI to be more general and to allow for more complex data fitting.

The probabilities for each class were computed and saved as a header for use on the microcontroller.

Filtering the Sensor Data

To enhance the accuracy of the classifier, I implemented a two-stage filtering process on the STM32 . The initial stage involved a basic moving average filter, followed by a Kalman filter in the second stage.

The OptiGap Sensor System Demo

The GIFs provided below illustrate various stages of the OptiGap sensor system, encompassing assembly and the operational demonstration of the final sensor system.

System Overview

Assembly of an OptiGap Sensor using TPU Filament

Attenuation of Light through the OptiGap Sensor

Fitting of the Sensor Data

Segment Classification using PMMA Optical Fiber

Segment Classification using TPU Filament

Underwater Operation

OptiGap Design Specifications

Key Properties & Parameters

Material Recommendations

Next Steps

I’ve made significant progress on the OptiGap system beyond what’s documented here, including its integration into another modular actuation and sensing system I developed called EneGate.

This has involved custom PCB design and systems integration, detailed in my dissertation. Additionally, I’ve prototyped miniature PCB versions of the optics to interface with the PCBs for the EneGate system.

I’ve also validated OptiGap on a real-world soft robotic system, with full details set to be presented in an upcoming RoboSoft paper titled “Embedded Optical Waveguide Sensors for Dynamic Behavior Monitoring in Twisted-Beam Structures.“

Commercialization

There’s an ongoing commercialization aspect to this research as well. Feel free to reach out if you’re interested in further details.

That’s it for now!

I don’t want to make this too long so I’ll end here. I hope this provided some insight into the research and development process involved in something like this. If you have any questions or would like to learn more, don’t hesitate to contact me!

The post R&D Case Study: Developing the OptiGap Sensor System appeared first on Paul Bupe Jr, PhD.

Update: This article was featured on Hackaday.com and on SDP/SI’s “Featured People and Organizations” page.

Goose is a mobile autonomous robot I designed and built over 6 months in my spare time for a robotics competition. This was a fully custom and challenging build that tested my competence in electrical engineering, mechanical engineering, control systems, and computer science.

Instead of focusing heavily on the competition, the goal of this article is to briefly go through the system design process. I’ll touch on my various design choices and discuss how I chose to address some of the common issues in designing an autonomous robot. This is not a tutorial (those will be coming later) but more of a case study.

Competition and Results

The basic premise of the competition was to have an autonomous robot navigate a square arena (enclosed by four walls) and collect / sort  multi-colored cubes and balls. It had to avoid fixed obstacles and had a 3-minute time limit. The robot was also required to “orbit” around the center of the arena in a counterclockwise fashion while performing the aforementioned tasks.

Goose was in second place after the first round then suffered a catastrophic reverse polarity condition at my hands (thanks Murphy) prior to the second run. I made two critical mistakes:

1. I daisy-chained the power rail going to my two motor controllers instead of tying each directly to my power bus. This set me up for the upcoming cascading failure.
2. After being awake for over 24 hour I somehow reverse polarized the power rail going to my motor controllers, which destroyed them both.

Even without a second round run, Goose did well enough in the first round to finish fourth overall in the Open competition.

Hardware Design

The first hardware decision I made was selecting the type of drive system the robot would employ. Since the robot was constrained to a 9″ x 9″ x 11″ box, and once I realized that I was just building a glorified vacuum cleaner, my design ideas converged towards a very compact Wall-E type chassis.

Based on experience I chose a tracked differential drive system, i.e., non-holonomic like a tank. As a general rule you want to avoid using omnidirectional wheels an autonomous robot unless you have a localization system that gives you an accurate pose. Using a two-motor differential drive system allowed me to achieve a compact design by placing the geared motors in the rear and directly driving the tracks.

SDP/SI was gracious enough to provide me with the timing belts and pulleys I used in my custom drivebase (shown in the image above) so thanks to them! I also would not have been able to build this robot without the amazing engineering/manufacturing facilities at Georgia Southern University that have state of the art equipment (such as the waterjet and laser cutter I used) available to students.

Prototyping and Base Build

After getting a rough idea of the physical design of the robot I started prototyping using SolidWorks, my 3D printer, and a laser cutter. My prototyping workflow involved first sketching out parts on paper, creating them in SolidWorks, then 3D printing or laser cutting them as needed. In total I ended up creating around 232 SolidWorks parts by the end of the build.

The biggest issue I ran into was figuring out how to remove play from the shaft in the bearing. The inner race of the bearing would move when under tension which caused alignment issues with the belt. The shaft was anchored at one end only because there was no room to add a second bearing (even a sleeve bearing) on the outer plate end due to the competition size constraints.

I was able to overcome this issue by designing a sturdy bearing mount that I mounted on the inside of the bracket then added a smaller acrylic mount with a sleeve bearing on the outside. This, along with shortening the shaft, was able to remove any play from the shaft under tension. I also added a very small lip to the pulleys to prevent the belt from slipping. The final base used aluminum which was machined using a MAXIEM 1515 waterjet cutter.

The remaining parts were created using wood with the help of a laser cutter.

Electronics

The electronics subsystem of Goose utilized an architecture that I have been tweaking for my autonomous robot designs over many years. The main idea is to minimize the cost of failure by isolating as many systems as possible and using generalized interfaces to connect the systems, i.e., modularization. All the supply rails are also individually fused with on/off switches and indicators.

Some core safety features of my architecture include:

• Proper fusing for overcurrent protection.
• Crowbar circuits for overvoltage protection. Crowbar circuits require very careful design in order to avoid false triggers due to noise or a narrow operating band.
• High current ideal diodes for reverse polarity protection. I did not include this in Goose and of course suffered a competition ending reverse-polarity condition so I won’t skip it again!

All these concepts are nothing new in electronics but striking the right balance between complexity and functionality for any one particular application can be tricky, especially when creating an autonomous robot.

Even though this system did not contain any sensitive analog circuitry, I still isolated the noisy motor power rail from the rest of the circuitry, eventually tying everything together at a single point at the battery in the “star ground” configuration. It’s always important to have a good idea of the major current paths in your circuits so you can spot any potential issues like ground loops — this does not require advanced knowledge of mesh analysis!

Mainboard

I have made it a habit to always use soldered PCBs for my projects. While breadboards are useful in the very initial stages of prototyping, they should not be used for anything past initial testing and brainstorming — they are too unreliable and can be difficult to troubleshoot due to issues like intermittent connections. Depending on your level of experience most analog circuits can be designed using a SPICE simulator and some basic math.

I designed the board for GOOSE using a stripboard. Stripboards are easier to use than regular solderable PCBs for more complicated circuits because they have connected strips (thus the name) that you cut to break the circuit at a desired point. This means you don’t have to solder a lot of holes in order to make a power bus or run a signal to multiple pins. Just don’t forget to cut the traces or you’ll have a pretty bad day!

The board contained the 5V and 3.3V power rails rated up to 2.5A, two PWM MOSFET driver circuits, a logic level converter, 9 DoF IMU, and the Teensy microcontroller. There are also a few other assorted supporting components.

Cabling and Wiring Harnesses

One of the most important and necessary skills in robotics is being able to make cables and wiring harnesses that are custom to the robot. While the process can be tedious, it results in a more reliable and professional end product. I typically use 2.54mm JST-XH connectors and “Dupont” connectors for my wiring when working with prototype boards. There’s quite a lot behind the names of connectors so I encourage you to read this article which does a great job summarizing most of what you’ll need to know about crimpers and connectors. For high current applications I used regular spade connectors and Wago 221 connectors for creating power busses. You can get generic versions of these on eBay and similar sites.

When creating long runs on this project I cannibalized a CAT5 cable and crimped on my own connectors to the ends. This was especially ideal because the twisted pairs meant I could effectively run differential signals if needed.

The NASA Standard for Crimping, Interconnecting Cables, Harnesses, And Wiring is a great resource for learning from the pros. While most of the crimping tools they use cost in the hundreds of dollars on the low end, the information is still relevant and quite useful!

Processing Subsystem

I used two processing cores for Goose: a Teensy 3.2 (32 bit ARM Cortex-M4) microcontroller to handle all the deterministic logic, and a Debian-based Raspberry Pi 3B+ for the more heavy computational work dealing with mapping and image processing. The Raspberry Pi was running the Robot Operating System (ROS) and the Teensy communicated to the Pi via USB serial.

Running at 96 MHz, the Teensy was fast enough to process data from two quadrature encoders and perform real-time odometry calculations; run an AHRS fusion algorithm from a 9 DoF IMU at about 2.1 kHz; run multiple PID loops; monitor two ultrasonic sensors at a high refresh rate, and send all this data to the Pi at a baud rate of 500,000. I had several ROS nodes running on the Teensy which communicated to the ROS Master on the Pi. As an example, I used the handler below to process messages from the Pi to the Teensy and then command the setpoint of the PID controllers:

void process_velocities(const geometry_msgs::Twist& cmd_msg)
{
	double x = cmd_msg.linear.x;	// forward velocity
	double z = cmd_msg.angular.z;	// rotational velocity

	if (z == 0) { // going straight
		left_motor_setpoint = x * 60 / (PI*wheel_diameter);
		right_motor_setpoint = left_motor_setpoint;
	}
	else if (x == 0) // turning in place
	{
		left_motor_setpoint = z * track_width * 60 / (wheel_diameter* PI * 2);
		right_motor_setpoint = -left_motor_setpoint;
	}
	else 
	{
		left_motor_setpoint = x * 60 / (PI * wheel_diameter) - z * track_width * 60 / (wheel_diameter * PI * 2);
		right_motor_setpoint = x * 60 / (PI * wheel_diameter) + z * track_width * 60 / (wheel_diameter * PI * 2);
	}
} // process_velocities()

Handler used to process velocity messages from the Pi and command the PID setpoints

I initially considered and prototyped using a BeagleBone Black since it contains two deterministic 32-bit PRU processing cores that can be accessed from the Linux system but I found the development overhead not worth it for one person with limited time. I wrote a few articles based on that experience here, here, and here.

Localization and Navigation Subsystem

The competition rules required the robot to move in large circles for the duration of the run which meant I couldn’t use encoder-based dead reckoning; the act of turning introduces errors due to drift and slippage which would accumulate to unacceptable levels over the course of the 3 minute run.

To counter this issue, I took advantage of the fact that the arena was enclosed by four walls in a square to develop a navigation system that used encoder odometry, IMU feedback, and LiDAR point cloud data to effectively pinpoint the location of the robot at all times.

As shown in the figure above, the basic idea was to use a pre-made map of the environment and then use point cloud data from the LiDAR in addition to the pose esitmate from to localize using a particle filter algorithm. I created the map in Adobe Illustrator and then converted it to a pgm file for use in ROS.

Localization inside a square would be difficult using LiDAR only (since all corners look the same and corner matching would fail) so I incorporated the pose estimate generated by fusing the encoder odometry data and the 9 DoF IMU using an unscented Kalman filter. I used an unscented Kalman filter (as opposed to the extended Kalman filter) for handling the nonlinear data because it is generally more accurate and has less computational complexity since there’s no need to calculate the Jacobian matrix.

Side note:

I have written a paper that includes some experimentation on developing a computationally efficient 2D navigation algorithm (for a microcontroller-based autonomous robot) which I might revise and release when I have the time. The algorithm I developed had a time complexity of O(n). Below is a page from the paper showing part of my experimental setup and a mapping / vector generated by the algorithm. I used a TFmini short-range LiDAR module for these experiments.

Final Thoughts

This project was certainly fun and a lot of work! There’s so much more to this robot I haven’t even touched on in this article but for the sake of keeping it “brief” I will have to end it here.

If time permits I may write a separate article on each system of the robot along with any associated source code and hardware info; possibly a series on building an advanced autonomous robot. I would like to go into more details on the software side and the workflow I came up with for working easily with ROS on a Raspberry Pi and a Teensy microcontroller.

Let me know your thoughts on this build or if you’d like to see any tutorials based on what’s in this build or robotics in general!

The post Designing an Advanced Autonomous Robot: Goose appeared first on Paul Bupe Jr, PhD.

ModBot is a simple robotics platform created for testing sensors, algorithms, vision systems, and everything else in between. I designed this platform with modularity in mind (thus the name) which requires the compartmentalization of behaviors and functions into discrete and, ideally, interchangeable modules. Since this is an experimental platform I opted not tie it to ROS (even though it still uses a Linux environment so ROS can be used) and created a very simple ASCII protocol for communication between modules. Lastly, I added teleoperation capabilities using a PlayStation DualShock 4 controller.

Robotics Platform Block Diagram and Design

The base is the fairly standard four-wheel differential drive design. A Raspberry Pi controls the motors via a dedicated Arduino Nano microcontroller that has motion profiles/velocity curves for each motor.

I utilized standard sockets and connectors so that controllers and other components could be swapped out without full disassembly of the robot. Notably, I used RJ45 Keystone jacks so that I can use standard Cat 5 cables as module interconnects. Multiple microcontrollers can be added to the system using the I2C protocol I created for communication.

All the core components of this robotics platform are kept under the upper deck, which is used to hold whatever component or sensor is being tested. The ports for the Raspberry Pi and other components are oriented such that cables can be plugged in or removed without removing the cover.

Serial Communication Protocol

To help facilitate modularity, I created a simple serial protocol to abstract the transfer of data between modules.

Why Create a Custom Protocol?

As I started building more complex systems with multiple “brains” working together (Raspberry Pis, BeagleBones, and Arduinos), I founding myself needing a simple protocol that could be used to more easily facilitate communication to all these controllers over I2C. Over various projects I standardized formalized the following protocol.

There are a lot of full-featured and extensive protocols and libraries out there (Firmata being at the top) and this does not attempt to replace or compete with those. It’s a simple single-purpose protocol with the sole focus of use for simple robotic systems with no overhead or fancy features.

Robotics Platform Communication Protocol

This I2C protocol is designed to be simple, human-readable, and is suitable for transmission over a serial bus. All data are passed as ASCII characters between 0x24 (36) and 0x7D (125). The protocol is simple enough that it can be debugged without any special decoding as show in the screenshot from my oscilloscope below.

This is just a basic overview and not meant to be full documentation of the protocol.

Packet Structure

<START><SRC_ADDRESS><DST_ADDRESS><ACTION><COMMAND><DATA><END><CHECKSUM>

• The START byte is ASCII “{” (123 decimal, 0x7B).
• The SRC_ADDRESS byte is any value from 64 to 95 decimal (0x40 to 0x5F, ASCII “@” to “_”).
• The DST_ADDRESS byte is any value from 64 to 95 decimal (0x40 to 0x5F, ASCII “@” to “_”).
• The ACTION byte is an ASCII “?” for GET and ASCII “$” for SET.
• The COMMAND bytes are three characters.
• The DATA bytes are variable length and are described in the Data section.
• The END byte is ASCII “}” (125 decimal, 0x7D).
• The CHECKSUM is calculated by subtracting 32 from all the characters in the packet (excluding the start and end bytes) and summing them. The modulo 95 of this value is then calculated and 32 is added back to that value.

Checksum Calculation

int calculate_checksum(String packet) {
   int sum = 0;
   int c = packet.length();
   for (int i = 0; i < c; i++) { sum += packet[i] - 32;}
   return (sum % 95) + 32;
}

Example Commands

The available commands for this revision are listed in the table below.

Example MTR GET Command and Response

MTR GET command. This packet requests the speed of motor number 2.

{@e?MTR02}X

MTR GET response. This packet returns the speed of motor number 2.

{e@?MTR02+078}C

Teleoperation Test Run

Here’s a short clip of me testing the motion profiles and mixing of the inputs from the controller joystick. As you can probably tell there was still a bit of tweaking to be done when I took this video but overall the robotics platform was quite responsive.

The post ModBot: A Modular Robotics Test Platform appeared first on Paul Bupe Jr, PhD.