Posts | Comments

Planet Arduino

Archive for the ‘rants’ Category

It is pretty well-known that I’m not a big fan of the Arduino infrastructure. Granted, these days you have more options with the pro IDE and Platform IO, for example. But the original IDE always gives me heartburn. I realized just how much heartburn the other day when I wanted to something very simple: increase the receive buffer on an ATmega32 serial port. The solution I arrived at might help you do some other things, so even if you don’t need that exact feature, you still might find it useful to see what I did.

Following this experience I am genuinely torn. On the one hand, I despise the lackluster editor for hiding too much detail from me and providing little in the way of useful tools. On the other hand, I was impressed with how extensible it was if you can dig out the details of how it works internally.

First, you might wonder why I use the IDE. The short answer is I don’t. But when you produce things for other people to use, you almost can’t ignore it. No matter how you craft your personal environment, the minute your code hits the Internet, someone will try to use it in the IDE. A while back I’d written about the $4 Z80 computer by [Just4Fun]. I rarely have time to build things I write about, but I really wanted to try this little computer. The parts sat partially assembled for a while and then a PCB came out for it. I got the PCB and — you guessed it — it sat some more, partially assembled. But I finally found time to finish it and had CP/M booted up.

The only problem was there were not many good options for transferring data back and forth to the PC. It looked like the best bet was to do Intel hex files and transfer them copy and paste across the terminal. I wanted better, and that sent me down a Saturday morning rabbit hole. What I ended up with is a way to make your own menus in the Arduino IDE to set compiler options based on the target hardware for the project. It’s a trick worth knowing as it will come in handy beyond this single problem.

The Issue: Arduino Serial Buffer Size Limit

I won’t bore you with the details about getting the board to work since you will only care if you have one. Details are available in a discussion on Hackaday.io, if you really want to follow it. But the upshot was that for XModem transfers, [Just4Fun] felt like the default Arduino serial buffer wasn’t big enough to be reliable. It did seem to work with the default 64-byte buffer, but XModem sends more data than that and it would be easy to imagine it getting overrun.

How hard can it be to update the buffer? In one way, it is trivial. In another way, it is very difficult because the tools want to help you so badly.

Tool Chain

The little computer project uses a real Z80 chip and uses an ATMega32A for almost all the support functions. It generates the clock, acts like a serial port, acts like a disk drive, and so on. However, the ATMega32 doesn’t directly have Arduino IDE support so you have to install a toolchain for it. The project called for MightyCore so that’s what I used.

The libraries for hardware serial were all set up using #define statements to allow you to adjust the buffer sizes. By default, if you haven’t set up anything, you get a default based on the amount of RAM your processor provides:


#if !defined(SERIAL_TX_BUFFER_SIZE)
#if ((RAMEND - RAMSTART) < 1023)
#define SERIAL_TX_BUFFER_SIZE 16
#else
#define SERIAL_TX_BUFFER_SIZE 64
#endif
#endif
#if !defined(SERIAL_RX_BUFFER_SIZE)
#if ((RAMEND - RAMSTART) < 1023)
#define SERIAL_RX_BUFFER_SIZE 16
#else
#define SERIAL_RX_BUFFER_SIZE 64
#endif
#endif

Making the Change

So this is easy, right? Just define those symbols before HardwareSerial.h loads. Uh oh. That file is loaded by Arduino.h. The IDE wants to add that to your program and it forces it to be first. There seems to be some IDE versions that check if you already included it so they don’t include it twice, but version 1.8.5 didn’t seem to do that. Maybe I can add some options in the preferences to pass to the compiler. Nope. Not via the IDE, anyway.

Not that I didn’t try a lot of things. It was tempting, of course, to simply change the core libraries. But that’s bad. You might want the defaults later. If you update the tool chain, you’ll lose your updates. I wanted to avoid that. Some people on the Internet suggested making a copy of the platform files and modifying those. Still not ideal.

Test Your Assumptions With Custom Error Reporting

I could tell things I tried were not working because I would put #if statements and #error statements temporarily in HardwareSerial.cpp. For example:

#if SERIAL_RX_BUFFER_SIZE==256
#error 256
#endif

Now if a compile causes an error 256, I know I was able to set the size. If not, then the system was resisting my changes.

Compromise: Adding Menu Options At the Board Level

I really wanted a way to make a change just in my project and set the serial buffer sizes. I failed at that. What I did do was make a modification to the boards.txt provided by Mighty Core. Yes, I will have to watch for upgrades overwriting my changes, but they are simple and it will be obvious that it is missing.

The reason it will be obvious is that I created a menu for the IDE that only appears when using ATMega32 for Mighty Core. This menu lets you select a few preset buffer sizes.

There were three parts to making this work:

  1. You have to tell the IDE you have a menu item and what it looks like.
  2. The new item needs to set some compiler options.
  3. Because the existing system also sets some compiler options, you have to make sure not to clobber them.

The first part is easy. The boards.txt file was (for me) in ~/.arduino15/packages/MightyCore/hardware/avr/2.0.5/boards.txt. Near the top there’s a list of menu keys and I added mine to the end:


# Menu options
menu.clock=Clock
menu.BOD=BOD
menu.LTO=Compiler LTO
menu.variant=Variant
menu.pinout=Pinout
menu.bootloader=Bootloader
menu.SerialBuf=Serial Port Buffers (RX/TX)

Next, I moved down the file and added my menu before the existing LTO menu option for the ATMega32:


32.menu.SerialBuf.disabled=Default
32.menu.SerialBuf.disabled.compilerSB.c.extra_flags=
32.menu.SerialBuf.disabled.compilerSB.cpp.extra_flags=

32.menu.SerialBuf.SB64=64/64
32.menu.SerialBuf.SB64.compilerSB.c.extra_flags=-DSERIAL_RX_BUFFER_SIZE=64 -DSERIAL_TX_BUFFER_SIZE=64
32.menu.SerialBuf.SB64.compilerSB.cpp.extra_flags=-DSERIAL_RX_BUFFER_SIZE=64 -DSERIAL_TX_BUFFER_SIZE=64

32.menu.SerialBuf.SB128=128/128
32.menu.SerialBuf.SB128.compilerSB.c.extra_flags=-DSERIAL_RX_BUFFER_SIZE=128 -DSERIAL_TX_BUFFER_SIZE=128
32.menu.SerialBuf.SB128.compilerSB.cpp.extra_flags=-DSERIAL_RX_BUFFER_SIZE=128 -DSERIAL_TX_BUFFER_SIZE=128

32.menu.SerialBuf.SB12864=128/64
32.menu.SerialBuf.SB12864.compilerSB.c.extra_flags=-DSERIAL_RX_BUFFER_SIZE=128 -DSERIAL_TX_BUFFER_SIZE=64
32.menu.SerialBuf.SB12864.compilerSB.cpp.extra_flags=-DSERIAL_RX_BUFFER_SIZE=128 -DSERIAL_TX_BUFFER_SIZE=64

32.menu.SerialBuf.SB256=256/256
32.menu.SerialBuf.SB256.compilerSB.c.extra_flags=-DSERIAL_RX_BUFFER_SIZE=256 -DSERIAL_TX_BUFFER_SIZE=256
32.menu.SerialBuf.SB256.compilerSB.cpp.extra_flags=-DSERIAL_RX_BUFFER_SIZE=256 -DSERIAL_TX_BUFFER_SIZE=256

32.menu.SerialBuf.SB25664=256/64
32.menu.SerialBuf.SB25664.compilerSB.c.extra_flags=-DSERIAL_RX_BUFFER_SIZE=256 -DSERIAL_TX_BUFFER_SIZE=64
32.menu.SerialBuf.SB25664.compilerSB.cpp.extra_flags=-DSERIAL_RX_BUFFER_SIZE=256 -DSERIAL_TX_BUFFER_SIZE=64

32.menu.SerialBuf.SB25632=256/32
32.menu.SerialBuf.SB25632.compilerSB.c.extra_flags=-DSERIAL_RX_BUFFER_SIZE=256 -DSERIAL_TX_BUFFER_SIZE=32
32.menu.SerialBuf.SB25632.compilerSB.cpp.extra_flags=-DSERIAL_RX_BUFFER_SIZE=256 -DSERIAL_TX_BUFFER_SIZE=32

Menu Structure

You can see that the 32.menu object groups all the items together for this processor. The next part is our menu key (SerialBuf). After that is a unique key for each memory item. It is important that you don’t reuse these. So, for example, if you have two SB64 keys only one is going to work.

If you stop at that key and put an equal sign you can assign the menu item the text you want to display. For example “Default” or “64/64.” You can also extend the key with a property and that property will be set if the option is active.

So, for example, if you select 256/256 then the compilerSB.c.extra_flags property will get set. I made that name up, by the way, and you’ll see why in a minute.

Peaceful Coexistence

There is no property called compilerSB.c.extra_flags. The correct property is compiler.c.extra_flags. However, the Mighty Core LTO option uses the same key. That’s why it was important that the new menu appears first and also that it sets a fake property. Then the LTO code needs a slight modification:


# Compiler link time optimization
32.menu.LTO.Os=LTO disabled
32.menu.LTO.Os.compiler.c.extra_flags={compilerSB.c.extra_flags}
32.menu.LTO.Os.compiler.c.elf.extra_flags=
32.menu.LTO.Os.compiler.cpp.extra_flags={compilerSB.cpp.extra_flags}
32.menu.LTO.Os.ltoarcmd=avr-ar

32.menu.LTO.Os_flto=LTO enabled
32.menu.LTO.Os_flto.compiler.c.extra_flags={compilerSB.c.extra_flags} -Wextra -flto -g
32.menu.LTO.Os_flto.compiler.c.elf.extra_flags=-w -flto -g
32.menu.LTO.Os_flto.compiler.cpp.extra_flags={compilerSB.cpp.extra_flags} -Wextra -flto -g
32.menu.LTO.Os_flto.ltoarcmd=avr-gcc-ar

The big change is that each set of flags adds to whatever the new menu set in its custom property. This way, all the flags get put into the correct property, compiler.c.extra_flags.

I set up error traps to catch all the cases to make sure they were being set right. In addition, after removing those traps, I could see my memory usage go up accordingly.

Customize

Of course, you can modify the parameters if you want something different. You could also use this trick to set other parameters before the Arduino.h file takes over. There’s some documentation about how to set up the platform definitions, including boards.txt.

It would have probably been better for me to make a custom boards.txt file with the same information in it but then I’d need to take the rest of Mighty Core with me. Instead, I just keep a copy of the file called boards.txt.custom and if my menu disappears, I just have to compare that file with the boards.txt file to see what changed.

Of course, if you don’t have to support people using the IDE, maybe just give it up. The Pro IDE is better, even if it does have some shortcomings. Plus there’s always Platform.io.

Lug
15

Speak with those who consider themselves hardcore engineers and you might hear “Arduinos are for noobs” or some other similar nonsense. These naysayers see the platform as a simplified, overpriced, and over-hyped tool that lets you blink a few LEDs or maybe even read a sensor or two. They might say that Arduino is great for high school projects and EE wannabes tinkering in their garage, but REAL engineering is done with ARM, x86 or PICs. Guess what? There are Arduino compatible boards built around all three of those architectures. Below you can see but three examples in the DUE, Galileo, and Fubarino SD boards.

Arduino DUE uses Atmel ARM Arduino Galileo uses Intel x86 Fubarino SD uses PIC32

This attitude towards Arduino exists mainly out of ignorance. So let’s break down a few myths and preconceived biases that might still be lurking amongst some EEs and then talk about Arduino’s ability to move past the makers.

Arduino is NOT the Uno

When some hear “Arduino”, they think of that little blue board that you can plug a 9v battery into and start making stuff. While this is technically true, there’s a lot more to it than that.

  1. An Arduino Uno is justanAVR development board.AVRs are similar to PICs. When someones says “I used a PIC as the main processor”, does that mean they stuck the entire PIC development board into their project? Of course not. It’s the same with Arduino (in most cases), and design is done the same way as with any other microcontroller –
    • Use the development board to make, create and debug.
    • When ready, move the processor to your dedicated board.
  2. What makes an Arduino an “Arduino” and not justan AVR but the bootloader. Thus:
    • An Atmega328P is an AVR processor.
    • An Atmega328P with the Arduino bootloader is an Arduino.
  3. The bootloader allows you to program the AVR with the Arduino IDE. If you remove the bootloader from the AVR, you now have an AVR development board that can be programmed with AVR Studio using your preferred language.

There Is No Special Arduino Language

Arduino "blink" sketch should run on any Arduino compatible board.
Arduino “blink” sketch should run on any Arduino compatible board.

Yes, I know they call them sketches, which is silly. But the fact is it’s just c++. The same c++ you’d use to program your PIC. The bootloader allows the IDE to call functions, making it easy to code and giving Arduino its reputation of being easy to work with. But don’t let the “easy” fool you. They’re real c/c++ functions that get passed to a real c/c++ compiler. In fact, any c/c++ construct will work in the Arduino IDE. With that said – if there is any negative attribute to Arduino, it is the IDE. It’s simple and there is no debugger.

The strength comes in the standardization of the platform. You can adapt the Arduino standard to a board you have made and that adaptation should allow the myriad of libraries for Arduino to work with your new piece of hardware. This is a powerful benefit of the ecosystem. At the same time, this easy of getting things up and running has resulted in a lot of the negative associations discussed previously.

So there you have it. Arduino is no different from any other microcontroller, and is fully capable of being used in consumer products along side PICs, ARMs etc. To say otherwise is foolish.

What is the Virtue of Arduino in Consumer Products?

This is Ask Hackaday so you know there’s a question in the works. What is the virtue of Arduino in consumer products? Most electronics these days have a Device Firmware Upgrade (DFU) mode that allows the end user to upgrade the code, so Arduino doesn’t have a leg up there. One might argue that using Arduino means the code is Open Source and therefore ripe for community improvements but closed-source binaries can still be distributed for the platform. Yet there are many products out there that have managed to unlock the “community multiplier” that comes from releasing the code and inviting improvements.

What do you think the benefits of building consumer goods around Arduino are, what will the future look like, and how will we get there? Leave your thoughts below!


Filed under: Arduino Hacks, Ask Hackaday, Hackaday Columns, rants


  • Newsletter

    Sign up for the PlanetArduino Newsletter, which delivers the most popular articles via e-mail to your inbox every week. Just fill in the information below and submit.

  • Like Us on Facebook