Upload speed a lot slower on Arduino IDE than using esptool.py

[dimitre]

Hello, I'm using the classic FTDI1232 to upload sketches to ESP-01, both using Arduino IDE and esptool, and the possible upload speeds are so different I was wondering how to automate the upload process outside Arduino IDE.

On Arduino IDE my max baud rate possible is 230400 and it takes a while to upload some medium projects while using esptool I get 12x more speed

esptool.py --baud 3000000 --port /dev/cu.usbserial-12345678 write_flash --flash_mode dio --flash_size detect 0x0 /Volumes/ofw/TEENSYATUAL/ESP/ESTILHACO7Ota/e8266.bin

and the result

esptool.py v2.5.0
Serial port /dev/cu.usbserial-12345678
Connecting....
Detecting chip type... ESP8266
Chip is ESP8266EX
Features: WiFi
MAC: 5c:cf:7f:94:6a:8a
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 3000000
Changed.
Configuring flash size...
Auto-detected Flash size: 1MB
Flash params set to 0x0220
Compressed 295024 bytes to 212380...
Wrote 295024 bytes (212380 compressed) at 0x00000000 in 1.8 seconds (effective 1306.8 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...

It would be great to have Arduino possible upload speeds improved. Thank you

[earlephilhower]

I didn't know the 8266 had a compressed upload stub like the 32. PRs are always appreciated. platform.txt has the command invoked and would be worth looking at to see if a simple replacement could be done.

One worry I see is that for Linux/Mac you can expect python to exist. On Windows, though, I don't think we install any interpreter and hence esptool*.py* may be some trouble.

[devyte]

I have several things to say here:

• @dimitre ignored the issue template.
• @dimitre 's request here seems to be to somehow allow other speeds in the IDE menu. I seem to remember mention that some OSs (Windows?) limit the options, hence the IDE doesn't show all speeds. Personally, I see as high as almost 1Mbps on my IDE under Ubuntu. I stick to 230K though.
• Writing at speeds above 230K or so is just asking for trouble. If it works for you, awesome, but unless you have a board with a certified flash chip, it's mostly luck. Flash issues due to high speed have been a source of headaches for a lot of users, especially with the typical chinese clones.
• Writing compressed would be very useful for flashing the SPIFFS image.

@earlephilhower does your comment mean you're thinking of replacing our current esptool-ck with esptool.py?

[minida28]

I am on Windows 10, but strange I don't have option for 230k baud rate...

I've been using 921600 baud rate for very long time ago and never have issues except occasionally failed upload. But if 3000000 speed is possible, it will be awesome :)

[earlephilhower]

...edit...looks like upload speed is used, so this is less good than I thought...

@devyte I'm not thinking of replacing the uploader, but if someone comes along with a PR to do it I think it's worth serious consideration (as long as it works cross-platform, etc. as well as the current setup). Too many other bits of ocean to boil already with File, String, WebServer, and probably ten other things.

[dimitre]

@devyte sorry I'll follow the template before submiting anything else.
Complementing, I'm in OS X and I can use only up to 230400. It would be great already if it worked at 921600.

[devyte]

I do see upload speeds in boards.txt.
@dimitre @minida28 feel free to experiment with different speeds. PRs are welcome, but please be aware that the boards.txt file is not meant to be edited directly. Instead, boards.txt.py should be edited, and boards.txt should be generated with it.

[d-a-v]

@dimitre @minida28 you can use more upload speed using the board generator (option --customspeed 230 (or more, see help)). Speeds are os-dependant but since windows has evolved, maybe some more can be enabled.

One worry I see is that for Linux/Mac you can expect python to exist. On Windows, though, I don't think we install any interpreter and hence esptool*.py* may be some trouble.

I think this is why we are still not using it, to keep the arduino KISS way.
But this is interesting enough to add an option for it in menu / board generator. I'm also opened to this.

[minida28]

@d-a-v may I beg your pardon, but where is the 'help' ..? I searched in the front page, doc, FAQ, but can't find it...

(or more, see help))

By the way, I just tried the quick and dirty way (editing boards.txt file directly), and added 3 new speeds: 1000000, 1152000 and 3000000.

None of them seem to work and it falls back to the default 115200 baud rate.

Example output using 1M speed:

G:\arduino-1.8.5\hardware\esp8266com\esp8266/tools/esptool/esptool.exe -vv -cd nodemcu -cb 1152000 -cp COM4 -ca 0x00000 -cf C:\Users\miq\AppData\Local\Temp\arduino_build_36368/BlinkWithoutDelay.ino.bin 
esptool v0.4.13 - (c) 2014 Ch. Klippel <ck@atelier-klippel.de>
warning: unsupported baud rate: 1000000, using 115200
Uploading 251936 bytes from C:\Users\miq\AppData\Local\Temp\arduino_build_36368/BlinkWithoutDelay.ino.bin to flash at 0x00000000
................................................................................ [ 32% ]
................................................................................ [ 64% ]
................................................................................ [ 97% ]
.......                                                                          [ 100% ]

[d-a-v]

#4478
:) Sorry, still have work to do

Start the board generator without option, you will get help:

$ ./tools/boards.txt.py
C:\...\boards-txt-directory> .\tools\boards.txt.py

boards.txt generator for esp8266/Arduino

usage: ./tools/boards.txt.py [options]

 -h, --help
 --lwip          - preferred default lwIP version (default 2)
 --led           - preferred default builtin led for generic boards (default 2)
 --board b       - board to modify:
 --speed s       - change default serial speed
 --customspeed s - new serial speed for all boards
 --nofloat       - disable float support in printf/scanf
 --noextra4kheap - disable extra 4k heap (will enable WPS)
 --allowWPS      - synonym for --noextra4kheap

 mandatory option (at least one):

 --boards        - show boards.txt
 --boardsgen     - replace boards.txt
 --ld            - show ldscripts
 --ldgen         - replace ldscripts
 --package       - show package
 --packagegen    - replace board:[] in package
 --doc           - shows doc/boards.rst
 --docgen        - replace doc/boards.rst
 --allgen        - generate and replace everything
                   (useful for pushing on github)

available serial speed options (kbps): 9 57 115 230 256 460 512 921 
available board names: generic(115k) esp8285(115k) espduino(115k) huzzah(115k) cw01(115k) espresso_lite_v1(115k) espresso_lite_v2(115k) phoenix_v1(115k) phoenix_v2(115k) nodemcu(115k) nodemcuv2(115k) modwifi(115k) thing(115k) thingdev(115k) esp210(57k) d1_mini(921k) d1_mini_pro(921k) d1_mini_lite(921k) d1(921k) espino(115k) espinotee(115k) wifinfo(115k) arduino-esp8266(115k) gen4iod(115k) oak(921k) wifiduino(921k) wifi_slot(115k) wiolink(115k) 

[minida28]

I see thank you @d-a-v :)

[minida28]

Ok I tested using esptool.py (still in Windows 10 via GIT bash) using @dimitre command above, still does not work (with 1000000, 11520000 & 3000000 speed).
But it works at 921600.

$ ./esptool.py --baud 1000000 --port COM4 write_flash --flash_mode dio --flash_size detect 0x0 H:/BlinkWithoutDelay/BlinkWithoutDelay.bin
esptool.py v2.5.1-dev
Serial port COM4
Connecting....
Detecting chip type... ESP8266
Chip is ESP8266EX
Features: WiFi
MAC: 5c:cf:7f:12:64:2b
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 1000000
Changed.
Configuring flash size...

A fatal error occurred: Timed out waiting for packet header

Note: I tested with NodeMcu v1.0 with onboard USB serial chip (Silicon Labs CP210x)

Is this related to the type of USB serial chip being used?