Update: You can follow the discussion of this article around the web on HN, Adafruit, Hackaday, and Hackster.io!

This article explores the research and development journey behind my new sensor system, OptiGap, a key component of my PhD research. I’m writing this in a storytelling format to offer insights into my decision-making process and the evolution leading to the final implementation. It should hopefully provide a glimpse into the sometimes-shrouded world of PhD research and may appeal to those curious about the process. For a deeper dive into technical specifics, simulations, and existing research on this subject, my dissertation is available online here.

What does it do?

In very general terms, this sensor is basically a rope that if bent can tell you where along its length you bent it. The fancy term for that is “bend localization.”

OptiGap’s application is mainly within the realm of soft robotics, which typically involves compliant (or ‘squishy’) systems, where the use of traditional sensors is often not practical. The name OptiGap, a fusion of “optical” and “gap,” reflects its core principle of utilizing air gaps within flexible optical light pipes to generate coded patterns essential for bend localization.

How the OptiGap Sensor System Started

The idea for OptiGap came about while I was experimenting with light transmission through various light pipes (optical cables) for use as a bend detection sensor. I was initially trying to see how I could effectively “slow down” light through the fiber…a seemingly straightforward task, right?

During this process, I attached a section of clear 3D printer filament (1.75mm TPU) to a piece of tape measure for an experiment and incidentally discovered that when I bent the tape measure (and filament) at the spot where the electrical tape was attached, there was a significant drop in light transmission. I hypothesized that this was because the sticky residue of the electrical tape was causing the filament to stretch, which in turn reduced the light transmission.

To verify this hypothesis, I attached a longer piece of TPU to a tape measure and began bending it at various points to observe how light transmission would change.

I wrote a small Linux I2C driver for the VL53L0X ToF sensor to run on a Raspberry Pi and push the data to a socket using ZeroMQ. I then created a rough GUI in Python to pull the sensor data from the socket and visualize the light transmission data in realtime, shown in the GIF below, which very quickly validated my hypothesis. This validation marked the “Eureka!” moment that sparked the eventual development of the OptiGap sensor.

The OptiGap Realization

I realized that since I could control where the light was being attenuated, I could use this to encode information about the position of the bend on the sensor. Using electrical tape was not a practical solution, so I started looking for a more reliable and consistent way to create these attenuations. This led me to the idea of cutting the filament and then reattaching it together using a flexible rubber (silicone) sleeve, leaving a small air gap, as shown in the image below.

The main working principle of the air gap is that translation and/or rotation of one light pipe face relative to the other changes the fraction of light transmitted across the gap. The greater the bend angle, the more light escapes across the gap. The resulting change in intensity of the optical signal can then be correlated with known patterns for use as a sensor.

The Big Idea

I then proceeded to test this idea by creating multiple air gaps in a row and bending the filament to measure the attenuation.

As depicted in the GIF below, the optical intensity decreases at each air gap, with a more noticeable decrease as the bend angle increases. This initial experimentation served as proof of concept, demonstrating the feasibility of the idea. It led to the formulation of my final hypothesis of utilizing a pattern of these air gaps to encode information regarding the sensor’s bending and employing a naive Bayes classifier on a microcontroller to decode the bend location.

This concept resembles the functionality of a linear encoder. Linear encoders gauge an object’s linear movement, typically comprising a slider rail with a coded scale akin to a measuring ruler and a sensing head that moves across this scale to read it. Linear (absolute) encoders emit a distinct code at each position, ensuring consistent identification of displacement.

The OptiGap system, functioning like an absolute encoder, would encode absolute positions using patterns of bend-sensitive air gaps along parallel light pipes, effectively serving as a singular fiber optic sensor.

Encoding the Bend Location using Inverse Gray Code

Absolute encoders commonly employ Gray code, a binary system where two successive values differ in only one bit. This property allows for various applications, including error checking. However, Gray code isn’t optimal for the OptiGap sensor system. Here, we aim for consecutive values to differ by the maximum number of bits to facilitate easier differentiation. This necessity gave rise to Inverse Gray code.

Inverse Gray code is a binary code where two successive values differ by the maximum (n-1) number of bits. To implement this, I simply create cuts in the filament wherever there’s a “1” in the Inverse Gray code sequence. This approach can scale to any bit number. For the prototype, I utilized 3 bits, providing 8 possible positions.

Visualization of the OptiGap Sensor System

The illustration below depicts the signal patterns of the OptiGap sensor system for each bend position using three fibers. By employing a naive Bayes classifier, the sensor system can discern bend positions based on signal patterns. The third graph represents actual sensor data from the prototype system, utilized for training the classifier on the microcontroller.

The OptiGap Prototype

I proceeded to construct a prototype of the OptiGap sensor system, utilizing 3 strands of clear TPU 3D printer filament, each featuring a distinct pattern of air gaps. The image below showcases the filament just before cutting, with the cut pattern indicated on a piece of tape.

For the prototype, I employed a commercial 3:1 fiber optic coupler to merge the light from the 3 strands into a single fiber optic cable, resulting in the completion of the sensor prototype, as depicted below.

This marked the final phase of validating the hypothesis and operational theory behind the OptiGap sensor.

Reducing the Physical Size

The initial prototype proved to be large and bulky, primarily due to the size of the 3D printer filament used. Drawing from previous experience, I recognized that PMMA (plastic) optical fiber offered a smaller and more flexible alternative suitable for this application. Consequently, I assessed 500, 750, and 1000 micron unjacketed PMMA optical fibers from Industrial Fiber Optics, Inc. for the sensor strands, resulting in a significant reduction in sensor size.

I conducted tests on all three types of fibers to evaluate their light transmission and flexibility. Among them, the 500 micron fiber emerged as the optimal choice overall, although all three exhibited sufficient flexibility for this application.

Reducing the Optical Transceiver Complexity

I decided to switch from using the complex VL53L0X ToF sensor to a simple photodiode and IR LED setup to reduce the complexity of the system and to increase modularity. This also allowed me to use a  microcontroller to read the sensor data, which was a significant improvement over the initial prototype.

I then created a demo system for the sensor based around an STM32 microcontroller and a photodiode/IR LED setup.

Realtime Machine Learning on a Microcontroller

The final stage in developing the OptiGap sensor system involved integrating a naive Bayes classifier onto the STM32 microcontroller to decode the bend location from the sensor data. I opted for a naive Bayes classifier due to its efficiency compared to if-statements or lookup tables, its capability to handle new or previously unseen data, and its potential for increased accuracy by considering relationships between multiple input variables.

Implementing the naive Bayes classifier proved to be relatively straightforward. This classifier is a probabilistic model based on applying Bayes’ theorem to determine how a measurement can be assigned to a particular class, with the class representing the bend location in this context. I utilized the Arm CMSIS-DSP library for the classifier implementation.

Fitting the Sensor Data

The initial step in integrating the classifier was to fit the sensor data to a Gaussian distribution for each air gap pattern. To expedite this process, I developed a Python GUI for rapid labeling and fitting of the data using GNB (Gaussian Naive Bayes) from the scikit-learn library.

I later improved this UI to be more general and to allow for more complex data fitting.

The probabilities for each class were computed and saved as a header for use on the microcontroller.

Filtering the Sensor Data

To enhance the accuracy of the classifier, I implemented a two-stage filtering process on the STM32 . The initial stage involved a basic moving average filter, followed by a Kalman filter in the second stage.

The OptiGap Sensor System Demo

The GIFs provided below illustrate various stages of the OptiGap sensor system, encompassing assembly and the operational demonstration of the final sensor system.

System Overview

Assembly of an OptiGap Sensor using TPU Filament

Attenuation of Light through the OptiGap Sensor

Fitting of the Sensor Data

Segment Classification using PMMA Optical Fiber

Segment Classification using TPU Filament

Underwater Operation

OptiGap Design Specifications

Key Properties & Parameters

Material Recommendations

Next Steps

I’ve made significant progress on the OptiGap system beyond what’s documented here, including its integration into another modular actuation and sensing system I developed called EneGate.

This has involved custom PCB design and systems integration, detailed in my dissertation. Additionally, I’ve prototyped miniature PCB versions of the optics to interface with the PCBs for the EneGate system.

I’ve also validated OptiGap on a real-world soft robotic system, with full details set to be presented in an upcoming RoboSoft paper titled “Embedded Optical Waveguide Sensors for Dynamic Behavior Monitoring in Twisted-Beam Structures.“

Commercialization

There’s an ongoing commercialization aspect to this research as well. Feel free to reach out if you’re interested in further details.

That’s it for now!

I don’t want to make this too long so I’ll end here. I hope this provided some insight into the research and development process involved in something like this. If you have any questions or would like to learn more, don’t hesitate to contact me!

The post R&D Case Study: Developing the OptiGap Sensor System appeared first on Paul Bupe Jr, PhD.

This post quickly covers how I disabled (removed) the IR LEDs on several Tapo C110 cameras I own. This was to use them behind a window without the IR LEDs’ glare ruining the video. The Tapo C110 is an excellent small WiFi security camera, boasting up to 2K resolution, local continuous SD card recording, and two local RSTP streams. Most importantly, it works perfectly fine without an internet connection once configured, fitting my use-case. A significant drawback of the camera is the need to completely disable the night vision mode to turn off the IR LEDs. This means you can’t use the night vision mode while the camera is behind a window. Thankfully its easy to disable Tapo camera IR LEDs!

The Official (lack of a) Solution

A discussion has been ongoing since 2021 on the TP-Link forum regarding this issue. The official TP-Link support recommends disabling night vision mode entirely. Meanwhile, the most popular suggestion among forum users involves covering the LEDs with tape. However, this is often ineffective due to light leaking around and through the tape since the LEDs are inset. My guess for why this hasn’t been addressed with a straightforward firmware update is that the camera’s design links the digital output controlling the IR filter to the IR LEDs. As a result, they cannot be independently deactivated, representing a significant design flaw if true!

Opening the Camera

Opening the camera is just a matter of using a prying tool along the front edge of the camera and the front plate will pop off — there are four clips, one along each side. It’s okay to use a bit of force, you won’t break it (don’t quote me on that). Conveniently, the LEDs are on a daughter board that connects to the PCB.

The Simple (slightly broken) Solution to Disable Tapo Camera IR LEDs

If you don’t mind losing the microphone and indicator LED you can simply remove the daughter board PCB entirely and the camera will still function. This allows the use of night vision mode without the IR LEDs causing reflections on the glass.

The Less Simple (more functional) Solution to Disable Tapo Camera IR LEDs

A more comprehensive approach involves desoldering the IR LEDs. This is relatively straightforward with tools like a hot air tool, heat gun, toaster oven, or even a stove and pan. I wouldn’t recommend using a soldering iron; the PCB’s heat sink properties mean that the LEDs may be damaged before the solder melts enough. I used a hot air tool on the PCB’s rear, directly behind an LED, and gently pulled on the LED with tweezers. The LEDs detached after a few seconds of heating. KEEP THE LEDs!

There are two hardware versions of the C110. Version 1 has three LEDs, while version 2 has two. I have one of each version, and found the LEDs in both easy to remove.

Reversing the Changes

If you decide to reverse these modifications, you can easily solder the LEDs back on or reattach the PCB. I ultimately reversed this change for all my cameras once they were no longer needed for window surveillance. Resoldering the LEDs is doable without extra solder, but I suggest using at least a bit of flux for better results. I personally applied a small dab of solder paste. For the resoldering process, I used a hot plate. It’s crucial to align the pads on the PCB with that on the LED by size to ensure correct polarity.

The post Disable TP-Link Tapo Camera IR LEDs appeared first on Paul Bupe Jr, PhD.

Over the past couple of years we’ve been working on a rather cool new class of robots called Soft Curved Reconfigurable Anisotropic Mechanism(s), or SCRAM, under this National Science Foundation EFRI project. For my latest paper, Electronically Reconfigurable Virtual Joints by Shape Memory Alloy-Induced Buckling of Curved Sheets, I spent a lot of time working with shape memory alloy (SMA) wire while developing a SCRAM device. Much of this time was spent trying to figure out exactly how to solder to SMA wire, specifically a nickel (Ni) – titanium (Ti) alloy called nitinol. My conclusion after all that time? Soldering to SMA wire is really hard!

Visualizing an SMA Wire Solder Joint

Soldering to SMA wire is hard due to the tough oxide layer on the surface. Solder beads off the wire as if trying to solder to glass! This can be seen in the two SEM images below (shout-out to the research facilities at the University of Louisville). I prepared two samples by tying some stranded wire around a piece of SMA wire (with the oxide layer ground off) and a piece of standard hook-up wire (tin-coated copper). I then applied some non-acidic flux followed by a healthy blob of tin/silver solder. Finally, I ground down one surface of each wire in order to perform some cross-sectional imaging.

It’s apparent from the images that there is no bond to the SMA wire (right) while the regular wire (left) has a great bond!

Soldering to SMA wire

The “easy” solution is to use a really aggressive acidic flux to eat away the oxide layer when soldering. The main disadvantage of this is the need to thoroughly wash the joint with water after soldering. Acidic flux will keep eating away long after soldering so it must be completely washed off! Another option is to use an inert gas as shielding while soldering (as in MIG welding) after grinding away the oxide layer. This again is not practical for most applications.

Here are some tips for soldering to SMA wire without acidic flux based on my aforementioned soldering adventures:

• Grind off the oxide layer first to improve wetting.
• Crimping is king. If possible, join wires with a crimp and then apply solder. This produces the most reliable connections.
• “Tie” the wire once around the SMA first to create a mechanical hold then apply solder.
• Use a silver alloy solder to increase the chances of proper wetting.
• Use the lowest temperature possible for the solder since heating up the SMA too high will cause an oxide buildup.

These techniques for soldering to stainless steel actually apply to SMA wire as well so they are worth studying.

If you enjoy research and things like this interest you then take a look at our lab page for some potential opportunities!

The post Soldering to Shape Memory Alloy (SMA) Wire is Really Hard appeared first on Paul Bupe Jr, PhD.

This tutorial covers setting up a pretty painless Raspberry Pi Pico C / C++ SDK development workflow on Windows using the Windows Subsystem for Linux (WSL) and Visual Studio Code (VS Code) with IntelliSense code completion!

With the Raspberry Pi Pico microcontroller being so new the current C / C++ SDK development process on Windows is a bit cumbersome. This tutorial should hopefully give you some ideas on how to go about programming the Raspberry Pi Pico the easy way with WSL and VS Code.

Note: This is not a beginner tutorial and assumes some minimal development experience.

The Raspberry Pi Pico is an exciting new microcontroller board launched on January 21, 2021. It’s based on the RP2040 microcontroller (by the Raspberry Pi Foundation) sporting a dual-core ARM Cortex-M0+ running at 133 MHz. The most exciting feature is the Programmable I/O, or PIO, that has 8 independent processors (simple state machines). This is huge for robotics where there’s a need to ingest real-time sensor data in the background while performing other tasks. Keep an eye out for my tutorial on reading multiple DC motor quadrature encoders at the same time without messing with lots of timers and interrupts using the PIO!

Prerequisites

1. Enable Windows Subsystem for Linux and install the latest Ubuntu image (20); follow the “Manual Installation” steps. WSL is a built-in feature of Windows 10 and brings most of the power of developing on a Linux machine natively into Windows! Be sure to install Windows Terminal for an even better experience (instructions towards the end of the link above).
2. Install Visual Studio Code and install the following extensions: Remote – WSL, C/C++, and CMake Tools. You can search for these extensions by name directly in VS Code.

Configure WSL

Setup Script

If you are in a bit of a rush I’ve created a setup script that will perform all the WSL configuration tasks for you. Just open up WSL, run the following commands, then skip to setting up Visual Studio Code!

cd ~
git clone https://github.com/74ls04/pico-wsl-setup.git
cd pico-wsl-setup
./pico_wsl_setup.sh

This script will install the SDK in ~/pico and also installs a few Pico extras from the Raspberry Pi Pico repos.

Install Dependencies

Note: I’m using Ubuntu 18 for this demo but if you’re starting from scratch use 20 to avoid issues with CMake compatibility.

First, open up the WSL Ubuntu terminal and go to the home directory.

cd ~

Then install the required build dependencies

sudo apt update
sudo apt install git cmake gcc-arm-none-eabi libnewlib-arm-none-eabi build-essential

That’s all there is for dependencies!

Install the Raspberry Pi Pico C / C++ SDK

While still in the home directory create a directory called pico and go into the pico directory

mkdir pico
cd pico

After that clone the SDK and examples git repositories.

git clone -b master https://github.com/raspberrypi/pico-sdk.git
cd pico-sdk
git submodule update --init
cd ..
git clone -b master https://github.com/raspberrypi/pico-examples.git

The git submodule command basically pulls in a separate Raspberry Pi repository from https://github.com/raspberrypi/tinyusb.git and downloads those files.

Here is what all the commands look like after execution.

Configure and Build with Visual Studio Code

Now we’re going to configure VS Code and build the following “blink” example.

#include "pico/stdlib.h"

int main() {
    const uint LED_PIN = 25;
    gpio_init(LED_PIN);
    gpio_set_dir(LED_PIN, GPIO_OUT);
    while (true) {
        gpio_put(LED_PIN, 1);
        sleep_ms(250);
        gpio_put(LED_PIN, 0);
        sleep_ms(250);
    }
}

So far you should have the following directories:

~/pico
~/pico/pico-sdk
~/pico/pico-examples

Move into the pico-examples directory

cd pico-examples

Now here is where the integration of VS Code and WSL really shines. Open VS Code into this directory

code .

This command will launch VS Code and automatically link it to the WSL instance. You should see a green connection label in the bottom left. This process also creates a hidden .vscode sub-directory that will hold the VS Code configuration for this workspace.

If this is your first time using VS Code with WSL follow these quick instructions to make sure the three extensions mentioned at the top are also installed in WSL. At this point you may see a notification asking if you would like to configure the project — click on “Not now” as we need to set up a few things first.

Configure CMake Extension

Click the gear icon in the bottom left and select Settings.

Expand the Extensions tree, select CMake Tools configuration, and then click Add Item under “Cmake: Build Environment” to add the SDK path as shown below. If you followed the directory structure in this tutorial your PICO_SDK_PATH will be /home/$USER/pico/pico-sdk where $USER is your WSL username. In this case my tutorial username is “main.”

Configure IntelliSense (Code Completions)

Open the Command Palette, Ctrl+Shift+P, and start typing “C/C++” then select C/C++: Edit Configurations (JSON).

This will create the following file:

 /home/$USER/pico/pico-examples/.vscode/c_cpp_properties.json

Change the file to match the following settings. Keep in mind that these settings are purely for code-completion and have nothing to do with the actual compiling.

{
    "configurations": [
        {
            "name": "Linux",
            "includePath": [
                "${workspaceFolder}/**",
                "/home/$USER/projects/pico/pico-sdk/**",
                "/usr/lib/gcc/arm-none-eabi/**"
            ],
            "defines": [],
            "compilerPath": "arm-none-eabi-gcc",
            "cStandard": "c11",
            "cppStandard": "c++17",
            "intelliSenseMode": "gcc-arm",
            "configurationProvider": "ms-vscode.cmake-tools"
        }
    ],
    "version": 4
}

Replace $USER with your WSL username under “includePath”!

Close VS Code and relaunch it again from the terminal with

code .

This time when CMake asks whether you’d like to configure the project select “Yes” and wait for it to finish configuring.

Configure Compiler and Build

Tell VS Code which compiler to use by clicking the “No Kit Selected” text at the bottom

Then selecting GCC for arm-none-eabi... If using Ubuntu 20 your compiler version will be different since I’m using 18.

Open up the “blink.c” example file using the explorer menu on the left, CTRL+SHIFT+E. You should see no errors highlighted in the file and code hints should be working as well.

Tell CMake which project to build by clicking on [all] at the bottom

Then scroll until you see “blink.”

You should now see [blink] instead of [all]. Click on “Build” to build the project!

VS Code will automatically create a build directory and spit out the project binaries into that directory.

Keep in mind this built the debug configuration, you can change it to release by clicking on CMake: [Debug]: Ready at the bottom.

Upload to Raspberry Pi Pico

Uploading the binary to the Pico is just a matter of dragging the .uf2 file into the folder that opens when you plug in the Pico while holding down the BOOTSEL button.

The file to transfer is:

/home/main/pico/pico-examples/build/blink/blink.uf2

To open the directory you can either right clock on the directory name in VS Code then click “Reveal in Explorer” or just type

explorer.exe .

in the WSL terminal to open that directory in Windows Explorer. That’s one of the most useful WSL commands!

Plug in the Raspberry Pi Pico while holding down the BOOTSEL button and then drag over the .uf2 file to the Pico folder.

It will automatically disconnect after transferring and immediately start running the program!

It’s also pretty easy to configure VS Code to automatically move the file after building but for the sake of keeping this short I’ll save that for another time.

More to come!

This should hopefully give you an idea of how the current development process is for the Raspberry Pi Pico. My goal is to create another robotics-focused Pico tutorial in the coming months starting from scratch and using the PIO — maybe in conjunction with a Raspberry Pi!

Feel free to ask any questions or offer suggestions in the comments!

Updated: 2/11/2021

The post Easy Raspberry Pi Pico Microcontroller C / C++ Programming on Windows appeared first on Paul Bupe Jr, PhD.

It’s not very well known that the Raspberry Pi has a built-in serial console that provides a fully working Linux terminal over serial. This allows for network-free headless access, including the absolutely necessary tab auto-complete! This feature is extremely useful for those projects that don’t use a screen or need network access. It’s also quite necessary when you are using a Pi as a wireless access point for a LAN without internet sharing.

Quick disclaimer: This article assumes basic familiarity with the Raspberry Pi and the basics of interfacing with serial devices. I’m using Windows to demonstrate but this works equally well on Linux and Mac.

Hardware

You’ll need a Raspberry Pi (of course) and a USB -to-Serial device (like this one from Adafruit).

Firstly, connect the RX of the USB-to-Serial device to Pin 8 of the Pi (physical Pin 8, not the GPIO pin number) and TX of the USB-to-Serial device to Pin 10 of the Pi.

Remember, for UART communication you do not match up TX <–> TX and RX <–> RX, it’s the opposite (TX –> RX and RX <– TX) since one device transmits and the other receives.

Finally, connect the ground of the USB-to-Serial device to Pin 6 of the Pi (ground).

NOTE: Do NOT connect the 5V power wire from the USB-to-Serial adapter to the 5V power pin of the Pi. This can backpower the Pi and destroy it. All that’s necessary is that the USB-to-Serial adapter and Pi share the same ground reference. Using the Adafruit adapter, the connections look like the picture at the top of the page.

Software

To enable the serial console just add the line:

enable_uart=1

to /boot/config.txt on the Pi. /boot/ is also mounted automatically on Windows so you can do this on a fresh image immediately after burning to the SD card (you may need to take it out and re-insert it first).

After connecting the USB-to-Serial adapter you can figure out the serial COM port using device manager.

The serial port needs to be configured with the settings below. Generally everything below but the baud rate are the default on most systems so that’s all you need to specify.

    Speed (baud rate): 115200
    Bits: 8
    Parity: None
    Stop Bits: 1
    Flow Control: None

In this case I’m using PuTTY to connect to the Pi as shown:

After connecting you should be met with a blank screen, simply press ENTER to send a character and you’ll see the familiar login prompt. Enter the login credentials and you’re in, just like you SSH’d in!

To demonstrate that this is a fully working terminal you can even launch the “graphical” raspi-config utility.

That’s it!

This should hopefully be a useful bit of info for the more advanced projects that don’t use the Pi with a screen and mouse/keyboard connected.

The post Raspberry Pi Headless Access Using Built-in Serial Console appeared first on Paul Bupe Jr, PhD.

Localization, in the context of robotics, is the process of determining the location of something within an environment. This article serves as a basic introduction to indoor localization, covering the commonly used techniques and technologies.

The most well known and widely used method of localization is GPS. Even with its widespread use, GPS is not suitable for all applications because it largely relies on line-of-sight (LOS) communication to GPS satellites. In indoor environments, these signals are greatly attenuated by walls and roofs, meaning that GPS often does not work indoors. Even when signals from at least four satellites are received indoors, the localization accuracy is too low to be useful. In lieu of GPS, a large number of indoor localization techniques and real-time locating systems (RTLS) have been developed. These can be grouped into two broad categories: radio-frequency (RF) based (wireless), and non-RF based (non-wireless).

Non-wireless techniques generally use cameras and sensors like Inertial Measurement Units (IMU) and laser distance finders (like LiDAR) for localization. Cameras can use markers or extract visual features from an environment to perform localization using various techniques including the very common technique known as simultaneous localization and mapping (SLAM). This technique involves building a map of an environment while at the same time localizing in that environment. Data from an IMU and laser scans can also be fused with the visual data to improve accuracy. These techniques are very computationally heavy, require specialized hardware, and are typically used in robotics or autonomous navigation applications (see Tesla, Boston Dynamics). This is the same type of localization technique I used in my last robot and I will cover this in more detail in a separate article.

Wireless techniques utilize technologies such as Wi-Fi, radio-frequency identification (RFID), Bluetooth, and ultra-wideband (UWB). These technologies are useful for indoor localization because they are already found in existing infrastructure and can be set up at very low cost. Wireless localization techniques can be classified as range-based, or range-free.

Ranged-based Indoor Localization

Range-based (or distance-based) techniques are more accurate than range-free and involve measuring the distance from the unknown node being localized to some fixed nodes with known locations, typically called anchors. Most range-based localization algorithms utilize one of the common measurement techniques which fall into two categories: angle-based and distance-based. The most common angle-based measurement technique is Angle of Arrival (AoA) and the main distance measurement techniques are Time of Arrival (ToA), Time Difference of Arrival (TDoA) and Received Signal Strength (RSS).

Angle-based Measurement

AoA measurement techniques calculate the angle (bearing) between the unlocalized node and a fixed anchor with a known location. These measurements are obtained using two main techniques that include (1) measuring the amplitude response of the receiving antenna and (2) measuring the phase response of the receiving antenna. The location of the unknown node is a line having a certain angle from an anchor node — this requires at least two nodes to calculate the position as shown in the figure below.

The accuracy of these measurements is affected by the directivity of the antenna and the environmental NLOS and multipath effects. Since AoA measures angles, it requires direct LOS between the receiver and transmitter because a reflected signal arriving at the receiver can be interpreted as coming from a completely different direction, which can result in very large errors in the measurement.

Distance-based Measurement

Time of Arrival

Time of Arrival (ToA) is a technique that calculates distance based on the measured time of arrival of a signal from a transmitting node to a receiving one. This is more formally referred to as a one way propagation time measurement. The primary drawback to this technique is that it requires perfect time synchronization between the clocks of the transmitter and receiver nodes; any difference between the two clocks can become a large error in the distance calculation. Assuming normal conditions (air as the medium and radio waves traveling at the speed of light) a small clock synchronization error of 1ns will relate to a distance measurement error of 0.3m.

One way of overcoming this issue is by measuring the round trip propagation time instead of the one way propagation. The first node sends a signal to a second node which in turn immediately sends that signal back to the first node and the distance is calculated using the round trip time. This removes the need for the transmitter and receiver to have synchronized clocks. The primary issue with this round trip method is the processing delay accrued from the second node receiving the signal then sending it back in turn. This delay is usually known and specified by the manufacturer (or during a calibration process) so it can be subtracted from the measurement at the first node.

Time Difference of Arrival

Time Difference of Arrival (TDoA) is another technique that measures propagation time but in this case the difference between the arrival time of a signal at two different fixed receivers is measured. This requires that the location of the two receivers are known and those two receivers also have synchronized clocks. Unlike ToA, there is no need for the clocks between the transmitter and receivers to be in perfect sync.

Received Signal Strength

There are two main methods of estimating distance using the Received Signal Strength (RSS): using the path loss log-normal shadowing model and RSS fingerprinting. Distance estimation using the path loss model is accomplished by measuring the signal attenuation as it propagates from the transmitting node to the receiving node. The relationship between distance and signal attenuation is heavily dependent on channel characteristics and as a result requires a very accurate propagation model in order to have acceptable results. The standard log-normal model used in this technique is as follows:

(1)  

with being the reference power at distance from the transmitter, being the path loss exponent measuring the rate at which the RSS decreases with distance, and being a zero mean Gaussian random variable with standard deviation which accounts for random showing effects.

Multilateration

Multilateration is a core technique for estimating the location of a node using the measured distances to multiple anchors. Traditionally, this is achieved using 3 anchors (for 2D localization) and is referred to as trilateration. The locations of the anchors is assumed to be known and the location of the unknown node is the intersection of the three circles with the center at the location of each anchor and radius equal to the measured distance to the unknown node.

An example of the relationship between the node and anchors is shown in the figure above. In practice the measurements aren’t always accurate so the circles don’t intersect at a single point, as shown below.

In this case with the locations of the anchors and the estimated distances between the anchors and node known, trilateration then becomes a problem of solving three nonlinear circle equations

(2)  

where , , and , are the coordinates of anchors 1, 2, and 3. These equations can be linearized into the form

(3)  

with

(4)  

(5)  

(6)  

which can be solved using the least squares method:

(7)  

Multilateration is not to be confused with triangulation which uses knowledge of the angles between the node and anchors to find the node-to-anchor distances using the law of sines.

Range-Free Indoor Localization

Range-free techniques use the relative positions of existing nodes, connectivity information, or detecting the proximity of the unknown node to fixed anchors with known locations (using RFID or Bluetooth beacons) to localize. They are simpler, cheaper, and more energy efficient than range-based algorithms at the cost of having low localization accuracy. Because of this, range-free algorithms are generally only useful when coarse locations are desired.

Connectivity-based

Connectivity-based localization works by checking if a node is connected to another node. If each node is aware of all the connected nodes then coarse locations can be determined by counting the hops between nodes using an algorithm DV-Hop. Another technique is simply detecting the presence of a node near a knwon beacon, typically using RFID or Bluetooth.

Profiling / Fingerprinting

Another type of range free localization is the use of RSS profiling or fingerprinting in order to overcome the inaccuracies of RSS-based distance measurements. Fingerprinting involves taking RSS measurements at various locations and building a map of those measurements and their position. Localization then becomes a matter of matching RSS measurements, not distances, to the map in order to localize. This type of localization is very well suited for machine learning.

Next Steps

This should hopefully have been a quick introduction to the concept of localization and the common techniques currently in use. The next article will include some code samples for implementing some of these techniques using Python.

The content of this article is based off the work I did for my Masters thesis.

The post Indoor Localization – An Introduction appeared first on Paul Bupe Jr, PhD.

I’ve been designing a 4-layer PCB for a product for a few months now and I finally got to the stage of assembling the prototype after the PCB and components arrived. Most of the passive components I used were 0603 surface mount parts so reflow soldering was the only viable option (if I wanted to maintain my sanity).

Since I don’t have a proper reflow oven, I decided to use the next best thing I own: No not my hot air rework station, a toaster oven!

There are, of course, plenty of guides and tutorials online for modifying a toaster oven for reflow soldering. They all involve a temperature PID loop to precisely control the heat profile for various solder types, components, etc,.

I really just wanted to see if I could just turn it on and see what happens. Spoiler Alert: It works.

I just preheated the toaster oven for a few minutes, stuck the PCB in there, and watched for the solder to start flowing. This took about 30 to 40 seconds. I wish there was more to this so I could do a more in-depth write-up but honestly that was it — nothing fancy and no precise timing or temperature profiles.

I gotta say, the final board is quite a beauty!

The Fine Print

Do this at your own risk! I did it because I’m just building a prototype and the final board will be professionally assembled. In addition, I know what I’m doing and I read the datasheets of all my critical components. You’ll notice that I did not put the electrolytic capacitor in the oven — I could not guarantee not to violate the very precise timings and temperatures specified in the datasheet. Also, please mind the fumes.

The post Reflow Soldering in Unmodified Toaster Oven? appeared first on Paul Bupe Jr, PhD.

Update: This article was featured on Hackaday.com and on SDP/SI’s “Featured People and Organizations” page.

Goose is a mobile autonomous robot I designed and built over 6 months in my spare time for a robotics competition. This was a fully custom and challenging build that tested my competence in electrical engineering, mechanical engineering, control systems, and computer science.

Instead of focusing heavily on the competition, the goal of this article is to briefly go through the system design process. I’ll touch on my various design choices and discuss how I chose to address some of the common issues in designing an autonomous robot. This is not a tutorial (those will be coming later) but more of a case study.

Competition and Results

The basic premise of the competition was to have an autonomous robot navigate a square arena (enclosed by four walls) and collect / sort  multi-colored cubes and balls. It had to avoid fixed obstacles and had a 3-minute time limit. The robot was also required to “orbit” around the center of the arena in a counterclockwise fashion while performing the aforementioned tasks.

Goose was in second place after the first round then suffered a catastrophic reverse polarity condition at my hands (thanks Murphy) prior to the second run. I made two critical mistakes:

1. I daisy-chained the power rail going to my two motor controllers instead of tying each directly to my power bus. This set me up for the upcoming cascading failure.
2. After being awake for over 24 hour I somehow reverse polarized the power rail going to my motor controllers, which destroyed them both.

Even without a second round run, Goose did well enough in the first round to finish fourth overall in the Open competition.

Hardware Design

The first hardware decision I made was selecting the type of drive system the robot would employ. Since the robot was constrained to a 9″ x 9″ x 11″ box, and once I realized that I was just building a glorified vacuum cleaner, my design ideas converged towards a very compact Wall-E type chassis.

Based on experience I chose a tracked differential drive system, i.e., non-holonomic like a tank. As a general rule you want to avoid using omnidirectional wheels an autonomous robot unless you have a localization system that gives you an accurate pose. Using a two-motor differential drive system allowed me to achieve a compact design by placing the geared motors in the rear and directly driving the tracks.

SDP/SI was gracious enough to provide me with the timing belts and pulleys I used in my custom drivebase (shown in the image above) so thanks to them! I also would not have been able to build this robot without the amazing engineering/manufacturing facilities at Georgia Southern University that have state of the art equipment (such as the waterjet and laser cutter I used) available to students.

Prototyping and Base Build

After getting a rough idea of the physical design of the robot I started prototyping using SolidWorks, my 3D printer, and a laser cutter. My prototyping workflow involved first sketching out parts on paper, creating them in SolidWorks, then 3D printing or laser cutting them as needed. In total I ended up creating around 232 SolidWorks parts by the end of the build.

The biggest issue I ran into was figuring out how to remove play from the shaft in the bearing. The inner race of the bearing would move when under tension which caused alignment issues with the belt. The shaft was anchored at one end only because there was no room to add a second bearing (even a sleeve bearing) on the outer plate end due to the competition size constraints.

I was able to overcome this issue by designing a sturdy bearing mount that I mounted on the inside of the bracket then added a smaller acrylic mount with a sleeve bearing on the outside. This, along with shortening the shaft, was able to remove any play from the shaft under tension. I also added a very small lip to the pulleys to prevent the belt from slipping. The final base used aluminum which was machined using a MAXIEM 1515 waterjet cutter.

The remaining parts were created using wood with the help of a laser cutter.

Electronics

The electronics subsystem of Goose utilized an architecture that I have been tweaking for my autonomous robot designs over many years. The main idea is to minimize the cost of failure by isolating as many systems as possible and using generalized interfaces to connect the systems, i.e., modularization. All the supply rails are also individually fused with on/off switches and indicators.

Some core safety features of my architecture include:

• Proper fusing for overcurrent protection.
• Crowbar circuits for overvoltage protection. Crowbar circuits require very careful design in order to avoid false triggers due to noise or a narrow operating band.
• High current ideal diodes for reverse polarity protection. I did not include this in Goose and of course suffered a competition ending reverse-polarity condition so I won’t skip it again!

All these concepts are nothing new in electronics but striking the right balance between complexity and functionality for any one particular application can be tricky, especially when creating an autonomous robot.

Even though this system did not contain any sensitive analog circuitry, I still isolated the noisy motor power rail from the rest of the circuitry, eventually tying everything together at a single point at the battery in the “star ground” configuration. It’s always important to have a good idea of the major current paths in your circuits so you can spot any potential issues like ground loops — this does not require advanced knowledge of mesh analysis!

Mainboard

I have made it a habit to always use soldered PCBs for my projects. While breadboards are useful in the very initial stages of prototyping, they should not be used for anything past initial testing and brainstorming — they are too unreliable and can be difficult to troubleshoot due to issues like intermittent connections. Depending on your level of experience most analog circuits can be designed using a SPICE simulator and some basic math.

I designed the board for GOOSE using a stripboard. Stripboards are easier to use than regular solderable PCBs for more complicated circuits because they have connected strips (thus the name) that you cut to break the circuit at a desired point. This means you don’t have to solder a lot of holes in order to make a power bus or run a signal to multiple pins. Just don’t forget to cut the traces or you’ll have a pretty bad day!

The board contained the 5V and 3.3V power rails rated up to 2.5A, two PWM MOSFET driver circuits, a logic level converter, 9 DoF IMU, and the Teensy microcontroller. There are also a few other assorted supporting components.

Cabling and Wiring Harnesses

One of the most important and necessary skills in robotics is being able to make cables and wiring harnesses that are custom to the robot. While the process can be tedious, it results in a more reliable and professional end product. I typically use 2.54mm JST-XH connectors and “Dupont” connectors for my wiring when working with prototype boards. There’s quite a lot behind the names of connectors so I encourage you to read this article which does a great job summarizing most of what you’ll need to know about crimpers and connectors. For high current applications I used regular spade connectors and Wago 221 connectors for creating power busses. You can get generic versions of these on eBay and similar sites.

When creating long runs on this project I cannibalized a CAT5 cable and crimped on my own connectors to the ends. This was especially ideal because the twisted pairs meant I could effectively run differential signals if needed.

The NASA Standard for Crimping, Interconnecting Cables, Harnesses, And Wiring is a great resource for learning from the pros. While most of the crimping tools they use cost in the hundreds of dollars on the low end, the information is still relevant and quite useful!

Processing Subsystem

I used two processing cores for Goose: a Teensy 3.2 (32 bit ARM Cortex-M4) microcontroller to handle all the deterministic logic, and a Debian-based Raspberry Pi 3B+ for the more heavy computational work dealing with mapping and image processing. The Raspberry Pi was running the Robot Operating System (ROS) and the Teensy communicated to the Pi via USB serial.

Running at 96 MHz, the Teensy was fast enough to process data from two quadrature encoders and perform real-time odometry calculations; run an AHRS fusion algorithm from a 9 DoF IMU at about 2.1 kHz; run multiple PID loops; monitor two ultrasonic sensors at a high refresh rate, and send all this data to the Pi at a baud rate of 500,000. I had several ROS nodes running on the Teensy which communicated to the ROS Master on the Pi. As an example, I used the handler below to process messages from the Pi to the Teensy and then command the setpoint of the PID controllers:

void process_velocities(const geometry_msgs::Twist& cmd_msg)
{
	double x = cmd_msg.linear.x;	// forward velocity
	double z = cmd_msg.angular.z;	// rotational velocity

	if (z == 0) { // going straight
		left_motor_setpoint = x * 60 / (PI*wheel_diameter);
		right_motor_setpoint = left_motor_setpoint;
	}
	else if (x == 0) // turning in place
	{
		left_motor_setpoint = z * track_width * 60 / (wheel_diameter* PI * 2);
		right_motor_setpoint = -left_motor_setpoint;
	}
	else 
	{
		left_motor_setpoint = x * 60 / (PI * wheel_diameter) - z * track_width * 60 / (wheel_diameter * PI * 2);
		right_motor_setpoint = x * 60 / (PI * wheel_diameter) + z * track_width * 60 / (wheel_diameter * PI * 2);
	}
} // process_velocities()

Handler used to process velocity messages from the Pi and command the PID setpoints

I initially considered and prototyped using a BeagleBone Black since it contains two deterministic 32-bit PRU processing cores that can be accessed from the Linux system but I found the development overhead not worth it for one person with limited time. I wrote a few articles based on that experience here, here, and here.

Localization and Navigation Subsystem

The competition rules required the robot to move in large circles for the duration of the run which meant I couldn’t use encoder-based dead reckoning; the act of turning introduces errors due to drift and slippage which would accumulate to unacceptable levels over the course of the 3 minute run.

To counter this issue, I took advantage of the fact that the arena was enclosed by four walls in a square to develop a navigation system that used encoder odometry, IMU feedback, and LiDAR point cloud data to effectively pinpoint the location of the robot at all times.

As shown in the figure above, the basic idea was to use a pre-made map of the environment and then use point cloud data from the LiDAR in addition to the pose esitmate from to localize using a particle filter algorithm. I created the map in Adobe Illustrator and then converted it to a pgm file for use in ROS.

Localization inside a square would be difficult using LiDAR only (since all corners look the same and corner matching would fail) so I incorporated the pose estimate generated by fusing the encoder odometry data and the 9 DoF IMU using an unscented Kalman filter. I used an unscented Kalman filter (as opposed to the extended Kalman filter) for handling the nonlinear data because it is generally more accurate and has less computational complexity since there’s no need to calculate the Jacobian matrix.

Side note:

I have written a paper that includes some experimentation on developing a computationally efficient 2D navigation algorithm (for a microcontroller-based autonomous robot) which I might revise and release when I have the time. The algorithm I developed had a time complexity of O(n). Below is a page from the paper showing part of my experimental setup and a mapping / vector generated by the algorithm. I used a TFmini short-range LiDAR module for these experiments.

Final Thoughts

This project was certainly fun and a lot of work! There’s so much more to this robot I haven’t even touched on in this article but for the sake of keeping it “brief” I will have to end it here.

If time permits I may write a separate article on each system of the robot along with any associated source code and hardware info; possibly a series on building an advanced autonomous robot. I would like to go into more details on the software side and the workflow I came up with for working easily with ROS on a Raspberry Pi and a Teensy microcontroller.

Let me know your thoughts on this build or if you’d like to see any tutorials based on what’s in this build or robotics in general!

The post Designing an Advanced Autonomous Robot: Goose appeared first on Paul Bupe Jr, PhD.

I recently had to design a 4-layer PCB and I finally decided to give KiCad another try (after hearing great things about version 5). I’ve known about KiCad for years and even tried it once a few years back but it never felt quite “there” yet. After spending a few months designing a 4-layer board I have to say it has certainly won me over.

I had to design a fairly complex PCB that included a switching supply, high currents, plenty of thermal vias, high frequency traces, and some mixed-signal work so I was really curious to see what KiCad could do.

One of the most useful features in my opinion is the ability to easily view an interactive 3D rendering of the board with no additional setup. The ability to hide the soldermask and view just the copper layer in the 3D view is a great sanity check on top of the DRC.

Another really really really great feature of KiCad is the ability to easily import any 3D model as an STL or wrl file and visually align and scale it to a footprint. This feature coupled with a site like 3D Content Central that allows you find 3D models from virtually any supplier for free makes accurate PCB design really stress free.

Another underrated KiCad “feature” I have come to love is just how easy it is to move or copy components around in the schematic editor without having to hold down a mouse button. I think this feature, coupled with the convenient shortcuts, saved me quite a bit of time.

Based on this experience I think I’ll definitely stick with KiCad for PCB design until the single license cost for Altium becomes more reasonable. Expect a few detailed KiCad tutorials from me in the near future!

The post 4-Layer PCB Design in KiCad 5: Quick Thoughts appeared first on Paul Bupe Jr, PhD.

I recently needed to build Python 3.6 from source on the BeagleBone Black for a robotics project and discovered that the build would always fail after running out of memory. Who could have figured that 512MB of RAM wasn’t enough to build Python from source?! While I could have set up cross-compilation and performed the heavy lifting on my computer with more resources, I figured this was a perfect problem to solve by adding swap memory to the BeagleBone Black.

What is a swap space?

At a very high level, a swap space (usually in the form of a partition) allows the system to move some of the “lower priority” memory pages (blocks of memory) to the partition or file so that higher priority tasks can use the memory. The combined size of the physical memory and swap space is called virtual memory. Gary Sims has a great primer on swap spaces over at linux.com

In this case, the BeagleBone Black only has 512MB of RAM so I’ll be adding ~1GB of swap space to my SD card in the form of a swap file. This will allow it to perform more memory-intensive tasks without running out of memory (at a cost of slower access times).

Creating a swap file

Creating a swap space is a fairly simple process that involves creating a file (and filling it with zeros) then telling the system to use that file as a swap space. Finally, the system has to be configured to load that file on startup.

Check if you already have a swap file

sudo swapon -s

In my case I already created the file so it’s shown. The assumptions is that you’re performing these steps because you don’t have a swap space.

Create the swap file

Use the dd utility to create a file with a fixed size and fill it with “zeroes”. In this case I’m moving 1000 “zeroes” from /dev/zero in block sizes of 1024k (1MB) into my file at /var/cache/swap/swapfile. This will result in a swap file of around 1 gigabyte. Keep in mind that the swap file is actually named “swapfile” with no extension.

sudo dd if=/dev/zero of=/var/cache/swap/swapfile bs=1024k count=1000

use chmod to change the file permissions of the new swap file.

sudo chmod 0600 /var/cache/swap/swapfile

Use the mkswap  command to create the swap area using our newly created swap file.

sudo mkswap /var/cache/swap/swapfile

Finally, enable the swap area with the swapon command

sudo swapon /var/cache/swap/swapfile

Configure the swap file to automatically mount on boot

Open up the /etc/fstab file with the nano editor.

sudo nano /etc/fstab

Append the following line to the file:

/var/cache/swap/swapfile    none   swap   sw   0   0

That is all there is to it for this specific case – adding swap memory to the BeagleBone Black is pretty straightforward!

The post Adding Swap Memory to the Beaglebone Black appeared first on Paul Bupe Jr, PhD.