ESP8266 SDK 2.0

 

Catching up with the latest ESP8266_NONOS_SDK_V2.0.0_16_07_19

– got into some issues.

Seems like Espressif integrated some time compatibility functions into libmain.a which i already had into the tree, most likely due mbed tls port

If you have millis(), micros(), mktime() and friends then you may need to comment them out.

There is one exception – open soruce xtensa gcc provides time() which is required by ANSI C so it is in  – platform must provide gettimeofday.
The problem is time() is in libmain from the SDK – so the workaround is to strip time from your libc and use the one in the SDK.

You’ll get an error like:

lib/libmain.a(time.o): In function `time’:
(.irom0.text+0x1dc): multiple definition of `time’
/…/esp-open-sdk/xtensa-lx106-elf/lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/lib/libcirom.a(lib_a-time.o):/…/esp-open-sdk/crosstool-NG/.build/src/newlib-2.0.0/newlib/libc/time/time.c:43: first defined here

To solve:

cd  esp-open-sdk/xtensa-lx106-elf/xtensa-lx106-elf/sysroot/lib/

cp libc.a libc.a.save

xtensa-lx106-elf-objcopy -N time libc.a

and this does the trick.

Firmware booted with sdk 2.0 and looks working.

Happy hacking!

 

ESP32 is comming

Espressif announced on 05/11/2015 the upcoming ESP32 chip:esp32ann

 

While the specs are impressive a lot of questions a rise :

  • Support – Espressif is a small company as far as we knew – would they be able to support both chips in the long term?
  • Compatibility with esp8266?  – The new ESP32 will use the RTOS based SDK and most APIs are said to be compatible with the esp8266 SDK.
  • Price – The price will be higher than esp8266  but not much higher.

Beta testing program is coming soon.

ESP8266 Secure CA Verification

While working on a secure cloud for the gang of the esp8266 based devices we are developing we need SSL – real and secure.

And this August is the month of the SSL issues , it seems , but thankful to the Espressifs’ quick support they are on the way out.

SSL Memory Leak

The latest SDK v1.3.0 introduced a bug that simply didn’t call the disconnect callback of esp connections under some circumstances and that in turn leaked memory.  It’s not clarified but the case was when you had a tcp listener and ssl connection after the ssl connection is over, your tcp listener connections did receive disconnect callbacks anymore.  You can get the fix from bbs.espressif.cn

Server Certificate Verification

Recent memory optimizations gave enough memory to turn on server certificate validation. But hit the next bug – it fails

Still in progress but you can watch the development of the resolving on the forum: SSL CA Issue

Current state is that if you do not provide the two level chain, i.e. certificate and issuer certificate the connections is established ok.

 

ESP8266 building OTA firmware for 2MB boards

During the past weeks i’ve worked on getting the FOTA upgrades work on the 2MB boards by Olimex.

The wonderful esp-link project by Thorsten von Eicken was a great example of  two things:

  1. How to concatenate the espfs filesystem image with the firmware images.
  2. How to properly write a new image to the flash.

It was a nice example to start with.

So after a lot of fiddling with  Makefiles, cgi routines and esptool  – i’ve finally got the OTA working.

Gotchas:

  • esptool.py modifies the images when writting – you have to pass the correct options for the flash split you want to use. Why on earth it modifies a already correctly build images with app_genbin.py, when not requested too is subject to be discussed in another post.
  • When using 512 KB or 1024 KB flash the two firmware images are both mapped at 0x40200000 so when using user2.bin you need correct offset + 0x8000 for the 512KB case.
  • When using 1024KB images you do not need offset – second MB of the flash is mapped at 0x40200000 too. So no need to have offset and essentially that makes the user1.bin same as user2.bin /not tested yet/. But this means doing less builds and if you plan to distribute upgrades less content.
ifeq ("$(ESP_FLASH_SIZE)","512")
build/eagle.esphttpd.v6.ld: $(SDK_LDDIR)/eagle.app.v6.ld
        $(Q) sed -e '/\.irom\.text/{' -e 'a . = ALIGN (4);' -e 'a *(.espfs)' -e '}'  \
            $(SDK_LDDIR)/eagle.app.v6.ld >$@
build/eagle.esphttpd1.v6.ld: $(SDK_LDDIR)/eagle.app.v6.new.512.app1.ld
        $(Q) sed -e '/\.irom\.text/{' -e 'a . = ALIGN (4);' -e 'a *(.espfs)' -e '}'  \
                        -e '/^  irom0_0_seg/ s/2B000/38000/' \
            $(SDK_LDDIR)/eagle.app.v6.new.512.app1.ld >$@
build/eagle.esphttpd2.v6.ld: $(SDK_LDDIR)/eagle.app.v6.new.512.app2.ld
        $(Q) sed -e '/\.irom\.text/{' -e 'a . = ALIGN (4);' -e 'a *(.espfs)' -e '}'  \
                        -e '/^  irom0_0_seg/ s/2B000/38000/' \
            $(SDK_LDDIR)/eagle.app.v6.new.512.app2.ld >$@

endif
ifeq ("$(ESP_FLASH_SIZE)","1024")
build/eagle.esphttpd.v6.ld: $(SDK_LDDIR)/eagle.app.v6.new.1024.app1.ld
        $(Q) sed -e '/\.irom\.text/{' -e 'a . = ALIGN (4);' -e 'a *(.espfs)' -e '}'  \
            $(SDK_LDDIR)/eagle.app.v6.new.1024.app1.ld >$@
build/eagle.esphttpd1.v6.ld: $(SDK_LDDIR)/eagle.app.v6.new.1024.app1.ld
        $(Q) sed -e '/\.irom\.text/{' -e 'a . = ALIGN (4);' -e 'a *(.espfs)' -e '}'  \
                        -e '/^  irom0_0_seg/ s/6B000/7B000/' \
            $(SDK_LDDIR)/eagle.app.v6.new.1024.app1.ld >$@
build/eagle.esphttpd2.v6.ld: $(SDK_LDDIR)/eagle.app.v6.new.1024.app2.ld
        $(Q) sed -e '/\.irom\.text/{' -e 'a . = ALIGN (4);' -e 'a *(.espfs)' -e '}'  \
                        -e '/^  irom0_0_seg/ s/6B000/7B000/' \
            $(SDK_LDDIR)/eagle.app.v6.new.1024.app2.ld >$@

endif
ifeq ("$(ESP_FLASH_SIZE)","2048")
build/eagle.esphttpd.v6.ld: $(SDK_LDDIR)/eagle.app.v6.new.1024.app1.ld
        $(Q) sed -e '/\.irom\.text/{' -e 'a . = ALIGN (4);' -e 'a *(.espfs)' -e '}'  \
            $(SDK_LDDIR)/eagle.app.v6.new.1024.app1.ld >$@
build/eagle.esphttpd1.v6.ld: $(SDK_LDDIR)/eagle.app.v6.new.1024.app1.ld
        $(Q) sed -e '/\.irom\.text/{' -e 'a . = ALIGN (4);' -e 'a *(.espfs)' -e '}'  \
                        -e '/^  iram1_0_seg/ s/8000/10000/' \
                        -e '/^  irom0_0_seg/ s/6B000/E0000/' \
            $(SDK_LDDIR)/eagle.app.v6.new.1024.app1.ld >$@
build/eagle.esphttpd2.v6.ld: $(SDK_LDDIR)/eagle.app.v6.new.1024.app2.ld
        $(Q) sed -e '/\.irom\.text/{' -e 'a . = ALIGN (4);' -e 'a *(.espfs)' -e '}'  \
                        -e '/^  iram1_0_seg/ s/8000/10000/' \
                        -e '/^  irom0_0_seg/ s/6B000/E0000/' \
                        -e '/^  irom0_0_seg/ s/40281010/40201010/' \
            $(SDK_LDDIR)/eagle.app.v6.new.1024.app2.ld >$@

endif

ifeq ("$(ESP_FLASH_SIZE)","512")
flash: all
$(Q) $(ESPTOOL) --port $(ESPPORT) --baud $(ESPBAUD) write_flash \
0x00000 "$(SDK_BASE)/bin/boot_v1.4(b1).bin" 0x01000 $(FW_BASE)/user1.bin \
0x7E000 $(SDK_BASE)/bin/blank.bin
endif

ifeq ("$(ESP_FLASH_SIZE)","1024")
flash: all
$(Q) $(ESPTOOL) --port $(ESPPORT) --baud $(ESPBAUD) write_flash \
0x00000 "$(SDK_BASE)/bin/boot_v1.4(b1).bin" 0x01000 $(FW_BASE)/user1.bin \
0xfc000 $(SDK_BASE)/bin/blank.bin
endif

ifeq ("$(ESP_FLASH_SIZE)","2048")
flash: all
$(Q) $(ESPTOOL) --port $(ESPPORT) --baud $(ESPBAUD) write_flash \
0x00000 "$(SDK_BASE)/bin/boot_v1.4(b1).bin" 0x01000 $(FW_BASE)/user1.bin
# 0x1fe000 blank.bin
# 0x1FC000 esp_init_data_default.bin

endif

I’ve added  ESP_FLASH_SIZE variable to build correct linker scripts and flash commands. I’m working on a template project which will be available on github. But you’ve got the idea of how to do it. One nice addition i’m working on is the ability to have different output directories.

The other minor modification you will need is to properly initialize EspFs in your user_main and link the espfs_img.o in your final linking step.

Now there are two upgrade scenarios – user initiated for consumer devices and automatic for industrial deployment. And the two transports for them HTTP and MQTT. HTTP was easy, MQTT is next – stay tuned.

 

 

IoT Security

Engineering Internet Of Things Secure network  for our upcoming IoT service.

Goals:

  • Security
  • Redundancy
  • End to End Security
  • Secure Firmware Upgrades

Threats:

  • DoS Attacks
  • DDoS Attacks
  • DNS spoofing /don’t even think of DynDNS/
  • Sniffing
  • man in the middle

Security:

  • Using a VPN -> takes the problem away and requires extra settup. Not user friendly. The only pro is that it can use an existing infrastructure if present. A Big NO
  • SSL -> expired certificates , heavy overhead, often exploited. Closed source library, may already be hacked. NO
  • AES all the Way. A big YES
  • DoS/DDoS prevention – redundant MQTT brokers, with dns/firmware fallback

So Why AES?

  • Can provide end to end encryption
  • Is openly developed standard – no hidden treasures
  • Suitable for encrypting messages – MQTT messages
  • Fast on modern processors and fast enough on the esp8266
  • AES can run on smaller chips too.
  • One big con is that the mqtt user name and password can be sniffed and used to connect to the cloud. But that’s easy solvable, if they don’t speak right, disconnect them and force password change.

Secure Firmware upgrades

Only local and only user initiated, user must see and check the result of the upgrade – Any other option introduces big risk for the system and the user.

Data – What to protect and what not?

It’s well know that all encryption is value versus time. So do you really want to hide what was the temperature at your house 5 minutes ago? – May be, if you are paranoid, but you definitely want to lock the access to your internet enabled door locks.

So all the actuators must be crypto protected – they do things. While sensors can be divided into two types – sensitive and non-sensitive. For example – house alarm state is sensitive, just like house human presence . But the outdoor temperature is not sensitive, you can get it N+1 ways.

 

 

ESP8266 using different flash sizes – FOTA and Download tool

Which goes where – when using a bootloader for FOTA.

The files:

  • master_device_key.bin –  Obtained from Espressif Cloud
  • esp_init_data_default.bin – Stores default RF parameter values
  • boot.bin – bootloader
  • user1.bin and user2.bin – user firmware
  • blank.bin – blank settings , flash to get default parameters

1. 512KB

  • master_device_key.bin 0x3E000
  • esp_init_data_default.bin 0x7C000
  • blank.bin 0x7E000
  • boot.bin 0x00000
  • user1.bin 0x01000
  • user2.bin 0x41000

2. 1024KB Flash

  • master_device_key.bin 0x3E000
  • esp_init_data_default.bin 0xFC000
  • blank.bin 0xFE000
  • boot.bin 0x00000
  • user1.bin 0x01000
  • user2.bin 0x81000

3. 2048KB Flash

  • master_device_key.bin 0x3E000
  • esp_init_data_default.bin 0x1FC000
  • blank.bin 0x1FE000
  • boot.bin 0x00000
  • user1.bin 0x01000
  • user2.bin 0x81000

4. 4096KB Flash

  • master_device_key.bin 0x3E000
  • esp_init_data_default.bin 0x3FC000
  • blank.bin 0x3FE000
  • boot.bin 0x00000
  • user1.bin 0x01000
  • user2.bin 0x81000

 

How to use 1MB and above flash on espressif forum describes the use of download tool.

Here is what download tool does:

Options

  • Crystal Frequncy choices:  40Mhz, 26Mhz,  24Mhz – For a 40M crystal , the booting uart tx baud is 115200,(74880 for 26m accordinglyCrystal Freq will be set to the BYTE[48] of esp_init_data_default.bin
  • SPI Flash speed: 0 = 40MHz , 1 = 26.7MHz,  2 = 20MHz,  f = 80MHz  stored in BYTE[3]&0f of the images
  • SPI flash read mode: 0 = QIO, 1 = QOUT, 2=DIO, 3=DOUT  set to the 3rd BYTE at flash address 0x0 so it is BYTE[2].
  • Flash size: Flash size is SET to the higher 4bit of 4TH BYTE at flash address 0x0 which is BYTE[3]>>4)&0xf in the image:
    0:512KB(256KB+256KB)  – 4Mbit
    1:256KB – 2Mbit
    2:1MB(512KB+512KB) – 8Mbit
    3:2MB(512KB+512KB) – 16Mbit
    4:4MB(512KB+512KB) – 32Mbit
    5:2MB-C1(1024KB+1024KB) – 16Mbit
    6:4MB-C1(1024KB+1024KB) – 32Mbit

tout as vdd33 – esp_init_data_default.bin BYTE[107]  = 0xff

tout as adc – esp_init_data_default.bin BYTE[107]  = VDD*10 /vdd > 1.8 and vdd < 3.6/

Low power options are left out for now.

To be continued …

Olimex ESP8266 module – MOD-WIFI-ESP8266-DEV

I need a bigger flash and a breadboard friendly board. Quick research lead to the following variants, beside soldering a bigger flash on the ESP-XX series modules.

  1. NodeMCU board’s that have bigger flash and fit on a breadboard but have regulators and usb 2 serial on them.
  2. ESP-XX module with adapter,  don’t like .
  3. MOD-WIFI-ESP8266-DEV

And the winner is Olimex’s – MOD-WIFI-ESP8266-DEV .

i got some of them:

oli6

Cardboard boxes and the modules within packed in real anti static bags.

antistatic-web

 

I’ve soldered the headers, with chip on top – looked natural. But the GPIO  silkscreen markings got hidden at the bottom of the module – i think it’s better for development to have them on top, shorted out text is OK.

oli3 oli4

There are  two rows left on both sides of the the breadboard.

oli1

So, let’s see what the extra flash is good for.

SSL – Is SSL the right solution for #IoT?

FOTA  – What about the FOTA upgrades security?

Going down the rabbit hole it seems that the MOST important and hard part of a IoT implementation is to get the security right. So the extra flash will present an opportunity to test to some ideas on how to do it. And we need it for the service we are building – it’s just vital to have and to have it right.

esp8266 UDP debugging

After a lots of lots of wasted hours  debugging an I2C driver it came out that the noise from the connected cheep USB2TTL converter is way too much. Would have lost months without the help of the new DS1054z scope i’ve got.

So i came up with this – a simple UDP debugging.

Simple socket to send udp messages.

dbg

Call dbg_udp_start(0) and you are ready to go. If you prefer not to flood your LAN with debug messages broadcasts then instead of 0 provide your workstation IP address.

Them usage is as simple as:

dbg_udp(“My debug message\n”);

You can go and wrap your os_printf or whatever calls to send via UDP too.

Viewing the log is done by any netcat alike.

nc -l -u -p 20202

It’s going to be handy when the UART is talking to other devices. You can even debug your devices over the internet this way.