Until recently, if you wanted to do SLAM (Simultaneous Location and Mapping) with LIDAR without a huge amount of coding work, you really only had one choice: ROS on a beefy computer.

This had two problems: 1) beefy computer (cost, size) 2) ROS (complexity, overhead, crazy difficult UI)

Good news: you don’t need either of them anymore. 

Simon Levy has recently updated his very efficient BreezySLAM python code (paper describing it is here) to support the new generation of cheap and powerful LIDAR and single-board computers, including:

• Slamtech series of LIDARs (A1, A2, A3) using the RPLidar Python library. I recommend the A1, which is just $99 and has a range of 12m with 8,000 samples per second. Run the rpslam.py example see it working
• Single-board computers. Although it will work on a Raspberry Pi 3, I recommend the Odroid XU4, which is just $80 and easily twice as fast and otherwise works the same under Linux
• Mini PCs. Atom-based x86 mini PCs cost just a little more than single-board computers and are easier to expand. BreezySLAM has been tested on this one ($116, including Windows, which you should ignore and run Linux instead!) and works fine.
• Alternatively, get a full-fledged Intel NUC, which is more expensive but can run BreezySLAM at higher resolution with lots of processing overhead to do other things like machine learning and computer vision. If you get this one ($339), you’ll need memory and storage 
• It will of course also work on any PC running Linux

If you combine an Odroid and the RP-Lidar A1, you’ve got a powerful full Lidar SLAM solution for less than $200!

A few notes:

• Follow the BreezySLAM instructions here. If you’re using a RP Lidar device (recommended!) don’t forget to install the RPLidar library first: “sudo pip3 install rplidar”
• Depending on the speed of your computer, you may need to sample fewer data points per update, which you can do by modifying the “MIN_SAMPLES” line in the code. For example, on the Atom Mini PC above I can only get 195 samples before I get the “no map screen of death” shown here:

• On the Intel NUC, I can get to 210 points per pass.
• On the Odroid I can get 200 points
• On the Atom Mini PC, I can get 195
• On the Raspberry Pi 3, I can only get 75 at one update/sec, with lots of artifacts, so not really useful yet.

My recommendation: go with the RP Lidar A1 and the Odroid. That’s less than $200 and it works great.

 

 

 

 

 

(Above, from left: Pixy 2, Jevois, OpenMV)

Computer vision used to be hard. No more. Now you can run OpenCV on a RaspberryPi, or even better use a dedicated computer vision camera/computer combo that costs less than $60. It’s kind of amazing, and perfect for small and cheap autonomous cars.

I’ve tested three of these camera/CV combo products with a DIY Robocar as the use case.

Here’s a quick review of the three and how they compare. I’ve previously compared Jevois and OpenMV, but both have improved since then — Jevois on the software side, and OpenMV on both the software and the hardware side with the forthcoming H7.  But now Charmed Labs has come out with the Pixy 2, which adds line following along with much improved hardware and software, so I’m expanding the comparisons.

For the OpenMV car, I tested the “Minimum Racer” configuration. For the other two, I used the Adafruit rover chassis, an Arduino and a motor driver board. My code for the Pixy 2 with the Adafruit motor driver shield is here 

Winner: OpenMV M7/H7

Why: Best IDE, copious I/O and shields mean no extra parts to drive a car, easy to program. Still the one to beat, and with the forthcoming H7 (to be launched via Kickstarter in September) it will have the computing power to do amazing frame rates and neural networks.

Fun Feature: The way it handles AprilTags (like QR codes, but super fast for robots to recognize) is miraculous. It can recognize them from 20 feet away when integrated with blob tracking.

Second Place: Pixy 2

Why: This is a big improvement over the original Pixy, which could just do color blob tracking. The new version has a fun line-following app and good Arduino/RaspberryPi integration. But it’s not really programmable and you’re limited to the four apps already loaded. And you need to add an Arduino and motor driver shield to drive a car. It doesn’t really do anything that OpenMV doesn’t do better.

Fun Feature:  The three white LEDs make for great headlights!

Third Place: Jevois

Why: The Jevois hardware is amazing, but it has been hamstrung by overly complex software (no IDE)  (UPDATE 6/20: It now has one and it looks great) and no I/O. The I/O problem remains, but the software has been much improved and now includes some good TensorFlow deep learning demos and some basic Linux command line support. But like the Pixy 2, to drive a car it needs an Arduino (or equivalent) and motor driver board to be connected.

Fun Feature: It can recognize 1,000 objects out of the box thanks to TensorFlow.

---

Here’s how they compare in features

	OpenMV M7/H7	Pixy 2	Jevois
Cost	$65 (M7), $59 (H7, coming in Sept)	$59	$59
IDE	Full-featured, cross platform (Mac, Win, Linux)	Just a viewer app	Full-featured, cross platform (Mac, Win, Linux)
Language	Micropython	Not really programmable	Python
Processing power (QVGA)	60 FPS (M7) 120 FPS (H7)	60 FPS	60 FPS
Memory	0.5MB RAM, 2 MB Flash (M7) 1MB RAM, 2MB Flash (H7}	N/A	256MB
Add-on boards available	Lots: servo, Wifi, motor controller, LCD	None	None
Interchangeable lenses	Yes	No	No
Interchangeable sensors	No (M7) Yes (H7)	No	Yes
Sample apps	~30 covering wide range including deep learning	4 (color and line tracking). No deep learning	~30 covering wide range including deep learning
I/O	USB, SPI, CAN, I2C, 3x PWM, Serial, ADC, DAC,	USB, SPI, 2x PWM	USB
SD Card	Yes	No	Yes
Lights	2x IR	3x White	No
Cost to make a car	$100 (OpenMV + motor shield + car)	$115 (Pixy 2 + Arduino + Adafruit Motor Shield + car)	$115 (Jevois, + Arduino + Adafruit Motor Shield + car)

If you want to use a cheaper motor driver board ($7) for the OpenMV-based “Minimum Rover“, here’s how to hook it up:

You’ll need some female-to-female hookup wires ($6).

First, plug four wires into the OpenMV’s GND, VIN, P7 and P8 pins as shown

Then connect those four wires to the motor controllers as shown in the picture below:

• P7 goes in In1
• P8 goes to In4
• GND goes to GND
• VIN goes to +5V

Then connect the motor wires to the terminals on each side of the motor controller, right motor wires to right side terminals, and left side motor wires to left side terminals, as shown. The wires from the battery go to the GND and 12V terminals (cut off the connector and put the bare wire in the terminal and screw it down).

The code requires no modification to use this motor controller. It should work exactly the same as the OpenMV motor controller shield.

 

 

After my first hands-on experiments with the new Benwake CE30-A solid state LIDAR (the version that generate an obstacle position rather than a point cloud), I thought I’d see if it was sufficient to drive a car through an obstacle field. It is. Above is my first effort at doing that, using a commonly-available Sunfounder RaspberryPi-based robocar.

My RaspberryPi Python code is here.

A few lessons:

1. You can run the Benwake CE30-A with the same 7.4v battery pack that you would use to drive the car. (It says it wants 12v but it works fine with less than that).
2. If you want to create a shorter cable. the connectors used by the CAN-to-USB converter board are JST-GH 1.25mm 4-pin connectors. You can buy them here.
3. You can 3D print a mount for the LIDAR. I’ve posted a 3D-printable file here.  Just raise it above the steering servo with some blocks of wood as shown here:

 

 

 

A year after announcing it, ST has finally released the new version of its tiny laser time-of-flight distance sensor with twice the range of the previous version. Called the VL53L1 (the previous version was the VL53L0), it’s available in chip form for $5 and in a breakout board from Tindie for $19 or a nicer one with connectors and mounting holes from Sparkfun for $25. These are chips originally designed for proximity sensors for smartphones (that’s how the phones can tell if you’re holding them up to your ear and dim/turn off the screen), and are very reliable. The claimed range is up to 4m indoors (up from 2m with the previous version), and since it’s a laser it has a very narrow beam (unlike sonar) and is appropriate for obstacle detection in DIY Robocars.

Sparkfun has just release an Arduino library for it, and the example code works well with both the Sparkfun and Tindie versions. If you want to try it, just connect either breakout boards SCL and SDA pins to the Arduino SCL and SDA pins and the voltage to the Arduino’s 3.3v pin and GND, as shown here:

In my testing so far, I’m only getting reliable reading up to 2.3m (7.5 feet), but there are some special high-power settings that you can enable, although they are not yet supported in the Sparkfun library.  That said, that’s more than enough for 1/10th scale robocars indoors, and is far more accurate than the broad-beam sonar or IR sensors that would otherwise be used for low-cost obstacle detection.

Outdoors, it’s pretty much useless, especially in bright sunlight. Expect no more than 1m outdoors even in the shade, so I’d recommend a different sensor such as sonar or a somewhat more expensive 1D lidar like the TFMini ($40) for that.

The era of small, cheap (sub-$1,000) Lidar is upon us, but it’s still a bit in its teething stage. In this post, I’ll give some initial hands-on impressions of one of the first solid-state 3D (actually closer to 2.5D) Lidars to hit the market, the Benewake CE30 series, which has just been released.

Unlike the other Lidars I’ve been using, such as the RP-Lidar A2/A3 series and (now discontinued) Scanse Sweep, which are rotating 2D Lidars (just viewing a thin horizontal disc around themselves), the CE30 has no moving parts and has both horizontal and (limited) vertical scanning (132° horizontal and 9° vertical), as well as an impressive 4m-30m range (depending on which version you get) as well as an excellent 20Hz refresh rate.  That’s perfect for small autonomous cars like ours — solid state means nothing to break in a crash, and having a vertical as well a horizontal sweep means that we can see obstacles from ground level to above the car.

Initial testing confirms the 4m range, dropping to about 3.5m in bright sunlight outdoors, which is quite good.

Official pricing ranges from less than $1,000 to $1,500, depending on the version, but at volume they’ll be available for $400-$500. If you want to buy one now in single units, you can find them on Roboshop: the CE30-A (USB version) is currently $999, the CE30-C (ethernet version) is $1,195 and the CE30-D (long range) is $1,499.

Here’s a table that shows how they compare:

	CE30-A	CE30-C	CE30-D
Max range	4m	4m	30m
Interface	USB/CAN	Ethernet	Ethernet
Notes	Only obstacle detection	Point cloud, no built-in obstacle detection	Point cloud. 4mm larger in width and height
Typical single-unit price	$800	$1,000	$1,500

Size-wise, it’s about the same size as the 2D Lidars, which is to say just right for our cars. (It’s shown above mounted on a Donkeycar, next to a Scanse Sweep for comparison).

I’ve posted a mount for it here, which you can 3D print for your car:

 

Here’s a screen-capture of the sort of data it provides (me in my workshop waving my arms like a dork). The top window is the uncorrected depth map and the bottom window is a top-down view.

The software support is still pretty minimalistic: a Windows demo program (shown above) and C++ libraries for Windows and Linux (including ROS). The Linux library is designed for x86 computers and won’t compile on a RaspberryPi (which is ARM-based) yet, but Benewake says that compatibility is coming soon. Stay tuned while I wait for that — I’m particularly interested in the built-in obstacle detection mode of the CE30-A, but want to run it on the RPi.

[Update: CE30-C Python code that runs on RaspberryPi is here. CE30-A Python code is here]

One of the tricky things if you’re using the CE30-C (ethernet interface) with a RaspberryPi is figuring out how to talk to the Lidar on Ethernet at the same time you’re connected to the Internet (or another network) over Wifi. After a lot of research and asking around, I finally figured it out. Raspian (the RaspberryPi Linux distro) has been changing its networking configuration process with each version, which makes it hard to find reliable tutorials online, but for the latest version (Raspian Stretch), edit your etc/dhcpcd.conf file to include these two lines:

This will assign your Ethernet port to the CE30, while leaving your Wifi adapter free to connect to your regular Wifi network.

Needless to say, if you opt for the USB version (CE30-A) you don’t need to deal with this — it just shows up as a serial port.

Finally, here’s Benewake’s promotional video that shows what these Lidar’s are really designed for.

Next post will be after I get the software up and running in an autonomous car stack. But so far so good!

The semi-monthly Oakland DIY Robocars races have now moved next door to the American Steel Poplar Gallery, which has a shiny, slippery polished concrete floor, so we’ve recalculated the performance metrics just to compare races on that track (as opposed to the rougher track at Pacific Pipe next door).  The data from the past four races are above (just the top five in each race), and you can see some clear trends:

1) An average race-to-race improvement of top finisher of about 10%:

2) Less of a gap between the top 5 cars, suggesting that people are starting to dial things in better

3) A machine learning car (Carputer) is currently in first place, but the next three are computer vision (most of them using OpenMV). That suggests that ML is potentially better, but harder to get working well, which agrees with our real-world experience.

4) At this pace, the autonomous cars will beat the best human time in the next 2-4 months. It’s worth noting we don’t actually know what the “best human time” is for this track because we don’t know any humans who can drive RC cars particularly well, so this is a guess based on the best driving we can do. Since the ML cars use “behavioral cloning” (the human drives while the car records the data, then the car attempts to do the same at 1.3-1.5x speed), the ML cars that use this technique are by definition faster than their masters. But that doesn’t mean that they could beat any human.

5) The really fun part is the “all cars demolition derby” at the end. Pay particular attention to the tiny car on the outside, which is a Minimum Viable Racer built by Eric. He was a good sport about following the Fight Club rules (“If this is your first night, you must race”) and we promised him we’d help him rebuild it if it got crushed. The race isn’t over until Eric is done!

 

 

It’s been almost exactly a year since we started the DIY Robocars communities, and the growth has been incredible. Here, in numbers, is what we’ve accomplished over the year.

• More than 20 race/hack events at our Oakland base. More than 2,000 total attendees.
• 1,200 members of our Oakland race/hack group
• 30 other DIY Robocars groups around the world
• A big DIY Robocars hackathon at Amazon’s AWS RE:Invent conference in Las Vegas
• Two big projects have come out of this:
  • Donkey (with 840 members of its Slack channel!)
  • OpenMV rover
• Race times are getting better!
• 

One of the problems of using RaspberryPi-based cars with Wifi connection is that even if you set them to auto-connect to a certain Wifi access point, you never know what their IP address will be, so you have to waste time search for them on the network. A good way to get around that (and to also display car information without having to SSH in via a terminal) is to mount a screen on the RaspberryPi, as shown above. I like the Adafruit 3.5″ TFT screen, which fits nicely on top of the Pi.

However, it’s not obvious how to get the screen to show your IP address once you connect, especially since the Wifi takes longer to connect than the Pi does to boot up, so any programs that run at bootup won’t be able to get the IP.  I searched for a while and then finally gave up and wrote it from scratch.  Here’s what you need to do:

First, edit the “crontab” file, which automatically runs programs at launch or at any frequency.  At the the command prompt, type “crontab -e”, which will ask you to select an editor. Select Nano, and add this line at the bottom of the file:

@reboot sleep 10; sudo python ip.py > /dev/tty1

Let me explain what that line does:

1. The “@reboot” part says to do this once after every reboot.
2. The “sleep 10;” part says to delay for ten seconds to give the Wifi time to connect.
3. The “sudo python ip.py” part says to run a Python script called ip.py, which we’ll add next
4. The “> /dev/tty1” part says to send the text output to the screen, which is called tty1.

Now we have to create the “ip.py” file that will display the IP address.  Type “sudo nano ip.py” to create the file and paste this:

import os
import socket
import fcntl
import struct

def get_ip_address(ifname):
 s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
 return socket.inet_ntoa(fcntl.ioctl(
 s.fileno(),
 0x8915, # SIOCGIFADDR
 struct.pack('256s', ifname[:15])
 )[20:24])

print("Wifi: ", get_ip_address('wlan0'))
print("Ethernet: ", get_ip_address('eth0'))

That’s it. Now the screen should show both the Wifi and Ethernet (if connected) addresses on bootup.

This is the cheapest good computer vision autonomous car you can make — less than $90! It uses the fantastic OpenMV camera, with its easy-to-use software and IDE, as well as a low-cost chassis that is fast enough for student use. It can follow lanes of any color, objects, faces and even other cars. It’s as close to a self-driving Tesla as you’re going to get for less than $100 😉

It’s perfect for student competitions, where a number of cars can be built and raced against each other in an afternoon.  NEW BONUS: If you want to move to a more advanced linear-regression code, instructions are here

Parts:

• OpenMV H7 ($65)
• Motor driver shield ($20)
• Robot car chassis ($13) (if that one is unavailable, any one that has two wheels and a castor will do)
• 3D Printed OpenMV mount ($12 from Shapeways. Free if you have your own 3D printer). If you want to modify it, the original files are on TinkerCad here.
• (If you don’t already have this): Some 22-gauge wire to extend the motor and battery wires to the camera ($6).
• Code

Optional:

• Polarizing filter ($10) to block sunlight glare
• Also, a cheaper motor driver board ($7)  can be substituted if you don’t mind a slightly less neat installation. For instructions on using that, see this note.

Total: $85 to $120, depending on which options you choose and whether you can 3D print your own parts.

The code is optimized for standard RGB tracks, which can be made with tape.

Instructions:

• 1) Cut two 12″ lengths of black and red wire pairs, and strip 1/4″ from each end. These will be the wires to connect your motors to the OpenMV board.
• 2) Assemble the rover kit as per the instructions, soldering the wire you just cut to the motor terminals (for consistency, solder the red wires to the terminals furthest from the chassis and the black wires to the terminals closest to the chassis, as shown in the picture below). We can always switch them at the motor driver side later, but this will make it easier not to get confused). Don’t overtighten the motor mounts; they’re fragile. If you break one, you can 3D print a replacement with this file, buy them from Shapeways here, or just cut it out of 3.5mm plywood.

• 3) As for the rocker on/off switch, just snap it into place on the chassis, then snip the red battery wire an inch from the battery case and solder that to one of the switch’s terminals, then solder the rest of the wire to the other terminal as shown here:

• 4) 3D print (or have printed at Shapeways) the camera mount. Attach it to the chassis with screws as shown in the pictures above.
• 5) Screw the OpenMV camera to the mount as shown:

• 6) Attach the motor and battery wires to the OpenMV motor shield as shown below. Once you’re done, carefully insert it into the connectors at the back of the OpenMV cam.

• 7) Load the code into the OpenMV IDE, plug your USB cable into the OpenMV board and run it while it’s looking at a green object (it defaults to following green, although that’s easy to change to any other color in the IDE). (Make sure your rover is powered on with batteries in). If one of the motors is turning backwards, just swap the wires from that motor going into the motor controller.
• Here’s how to test it and ensure it’s working:

• 8) If the rover tends to turn one way or the other, you can correct the “center” position by modifying the value in this line:

steering_center = 30  # set to your car servo's center point

• 9) Once you’re happy with the way the code is working, you can load it so it will start automatically even if a USB cable is not connected by selecting “Save open script to OpenMV Cam” in the Tools menu, as shown:

---

Code tweaking tips

If you want it to follow a different color, just change this number in the code:

threshold_index = 1
# 0 for red, 1 for green, 2 for blue

If you want to tune it for another color, or adjust it so it follows the color you’ve selected better for the specific tape and lighting you’ve got, use the IDE’s built-in Threshold Editor (Tools/Machines/Vision/Threshold Editor) and add a threshold set for the color (or replace one of the generic thresholds) that you want in this section of the code:

thresholds = [(0, 100, -1, 127, -25, 127), # generic_red_thresholds
              (0, 100, -87, 18, -128, 33), # generic_green_thresholds
              (0, 100, -128, -10, -128, 51)] # generic_blue_thresholds
# You may pass up to 16 thresholds above. However, it's not really possible to segment any
# scene with 16 thresholds before color thresholds start to overlap heavily.

In the below example, I’ve tuned it to look for red lanes. So I’d copy the “(0,100,30,127,-128,127)” and replace the generic red threshold numbers above with that.  Then I’d change the line above that to “threshold_index = 0”, so it would look for the first threshold in the list, which is red (lists are “zero-based”, so they start at zero).

When you’re done, the IDE will show it tracking a color lane like the above (the lowest rectangular “region of interest” — ROI — is weighted highest, with the other two weighted less). You can modify the ROIs by dragging boxes on the screen to show the region you want to identify, and then give it a weighting as show here:

# Each ROI is (x, y, w, h). The line detection algorithm will try to find the
# centroid of the largest blob in each ROI. The x position of the centroids
# will then be averaged with different weights where the most weight is assigned
# to the ROI near the bottom of the image and less to the next ROI and so on.
ROIS = [ # [ROI, weight]
(38,1,90,38, 0.4),
(35,40,109,43,0.2),
(0,79,160,41,0.6)
]