lock/unlock bits fail to verify with Atmel AVRISP mkII and Arduino AVR Boards 1.6.12/avrdude 6.3.0-arduino2

[per1234]

Tools > Board > ATmega1284P
Tools > Programmer > AVRISP mkII(MightyCore)/AVRISP mkII(they are equivalent for Burn Bootloader)
Tools > Burn Bootloader fails with:

C:\Program Files (x86)\arduino-1.6.10\hardware\tools\avr/bin/avrdude -CC:\Users\per\AppData\Local\Arduino15\packages\MightyCore\hardware\avr\1.0.6/avrdude.conf -v -patmega1284p -cstk500v2 -Pusb -e -Ulock:w:0x3f:m -Uefuse:w:0xfd:m -Uhfuse:w:0xd6:m -Ulfuse:w:0xf7:m 

avrdude: Version 6.3, compiled on Jun 22 2016 at 16:05:21
         Copyright (c) 2000-2005 Brian Dean, http://www.bdmicro.com/
         Copyright (c) 2007-2014 Joerg Wunsch

         System wide configuration file is "C:\Users\per\AppData\Local\Arduino15\packages\MightyCore\hardware\avr\1.0.6/avrdude.conf"

         Using Port                    : usb
         Using Programmer              : stk500v2
avrdude: usbdev_open(): Found AVRISP mkII, serno: 000200217496
         AVR Part                      : ATmega1284P
         Chip Erase delay              : 55000 us
         PAGEL                         : PD7
         BS2                           : PA0
         RESET disposition             : dedicated
         RETRY pulse                   : SCK
         serial program mode           : yes
         parallel program mode         : yes
         Timeout                       : 200
         StabDelay                     : 100
         CmdexeDelay                   : 25
         SyncLoops                     : 32
         ByteDelay                     : 0
         PollIndex                     : 3
         PollValue                     : 0x53
         Memory Detail                 :

                                  Block Poll               Page                       Polled
           Memory Type Mode Delay Size  Indx Paged  Size   Size #Pages MinW  MaxW   ReadBack
           ----------- ---- ----- ----- ---- ------ ------ ---- ------ ----- ----- ---------
           eeprom        65    10   128    0 no       4096    8      0  9000  9000 0xff 0xff
           flash         65    10   256    0 yes    131072  256    512  4500  4500 0xff 0xff
           lock           0     0     0    0 no          1    0      0  9000  9000 0x00 0x00
           lfuse          0     0     0    0 no          1    0      0  9000  9000 0x00 0x00
           hfuse          0     0     0    0 no          1    0      0  9000  9000 0x00 0x00
           efuse          0     0     0    0 no          1    0      0  9000  9000 0x00 0x00
           signature      0     0     0    0 no          3    0      0     0     0 0x00 0x00
           calibration    0     0     0    0 no          1    0      0     0     0 0x00 0x00

         Programmer Type : STK500V2
         Description     : Atmel STK500 Version 2.x firmware
         Programmer Model: AVRISP mkII
         Hardware Version: 1
         Firmware Version Master : 1.23
         Vtarget         : 5.2 V
         SCK period      : 8.00 us

avrdude: AVR device initialized and ready to accept instructions

Reading | ################################################## | 100% 0.00s

avrdude: Device signature = 0x1e9705 (probably m1284p)
avrdude: erasing chip
avrdude: reading input file "0x3f"
avrdude: writing lock (1 bytes):

Error while burning bootloader.
Writing | ################################################## | 100% 0.01s

avrdude: 1 bytes of lock written
avrdude: verifying lock memory against 0x3f:
avrdude: load data lock data from input file 0x3f:
avrdude: input file 0x3f contains 1 bytes
avrdude: reading on-chip lock data:

Reading | ################################################## | 100% 0.00s

avrdude: verifying ...
avrdude: verification error, first mismatch at byte 0x0000
         0xff != 0x3f
avrdude: verification error; content mismatch

avrdude done.  Thank you.

The explanation of the cause of this issue is described at arduino/avrdude-build-script#2

This issue will likely also occur for all MightyCore boards, all but ATmega16 have been modified in the patch for this issue, I don't find an entry for ATmega16 in the stock avrdude.conf file. I only have ATmega1284P to test with.

The solution which will still offer backward compatibility with previous Arduino AVR Boards versions requires defining an avrdude tool in the MightyCore Boards Manager JSON file. If you use avrdude 6.0.1-arduino5 then no further changes are required. If you use avrdude 6.3.0-arduino2 then the avrdude.conf patch found at https://savannah.nongnu.org/patch/index.php?8996 will need to be applied and unlock_bits and lock_bits values in boards.txt must be updated. The avrdude tool will also need to be installed for manual installation.
EDIT: platform.txt will also need to be modified to point to MightyCore's avrdude tool.

Another solution would be to just drop support for AVRISP mkII. The Atmel AVRISP mkII is no longer being made but there are still a lot of them around. The Olimex AVRISP mkII is still in production. I have only tested this issue with Atmel AVRISP mkII.

[per1234]

Same issue with MiniCore(tested with ATmega328P and ATmega168 but all include MCUs are in patch 8996).

Probably same issue with MegaCore(untested because I don't have the microcontrollers but the MCUs are in the patch)

Probably same issue with MicroCore(untested because I don't have the microcontroller but the MCU is in the patch)

[MCUdude]

Great! Now how do I apply this .patch file to the existing avrdude.conf files? Should there be a simple shell script to do all this, is the file executable from can the file be executed using avrdude or is this a copy/paste job?

[MCUdude]

I've actually experienced the exact same issue with an STK500 starter kit i got from the university. Uploading worked fine if I used the regular stk500 protocol, bur returned the ´0xff != 0x3f´ error if i used stk500pp.. I reported a bug on this last year but there was no response

[per1234]

I patched Arduino AVR Boards 1.6.12's avrdude.conf with patch 8996 here: https://github.com/arduino/avrdude-build-script/files/392783/avrdude.conf.txt. I had to add the .txt at the end so GitHub would let me attach it to the issue.

This is what I did to apply the patch on Windows, should be similar on other OS but will need a different patch tool. I think git can do it. This was my first time using a patch file so there's probably a better way:

1. Download https://savannah.nongnu.org/patch/download.php?file_id=37105
2. Download and install http://gnuwin32.sourceforge.net/downlinks/patch.php
3. Put the patch file and the avrdude.conf file to be patched in the same folder
4. Rename avrdude.conf avrdude.conf.in
5. Create a file named NEWS
6. Create a file named ChangeLog
7. Run this command: patch --input=no-lock-byte-read-mask.patch --binary --output=avrdude.conf

The --binary option was required because of the line endings.

The patches of NEWS and ChangeLog files will fail, but those files needed to be there for the patch to continue.

There were some changelog comments at the top of avrdude.conf that didn't patch correctly, probably because the patch is kind of old. Those should be doable by hand.

[MCUdude]

I don't think I need to install GNUwin32, since I'm using a mac and got a full blown Unix terminal (with the patch command already there). I'm not home right now, but I'll have a look at this in the evening (should be in the middle of the night where you live).

[per1234]

I agree, you definitely don't need to install the windows patch tool. I was just listing the process I followed. It's actually the middle of the night right now here(California, USA) but I'm a night owl. Hopefully it will add support for your starter kit also. I think that was the idea behind the change in AVRDUDE's behavior and this patch, to make the handling of the unused bits consistent. It's definitely causing a lot of breakage(see also arduino/Arduino#5175) but hopefully will be an improvement in the long term.

[MCUdude]

Will this patch break backwards compability with older IDEs that uses avrdude 6.0.1?

[per1234]

The patched avrdude.conf from Arduino AVR Boards 1.6.12 does break backwards compatibility:

avrdude: syntax error at C:\Users\per\AppData\Local\Arduino15\packages\MightyCore\hardware\avr\1.0.6/avrdude.conf:1071

but that's because line 1071 is:

usbpid = 0x2110, 0x2140;

Apparently that syntax is not compatible with avrdude 6.0.1-arduino5 but the patch doesn't add any of that, it only changes the lock bit read masks. If I copy the patch ATmega1284P lock bit read mask for ATmega1284P:

"x x x x  x x x x   o o o o  o o o o";

to your avrdude.conf line 2430 and change the unlock_bits and lock_bits values for ATmega1284 in your boards.txt to:

1284.bootloader.unlock_bits=0xff
1284.bootloader.lock_bits=0xcf

which, if I understand correctly, are equivalent to the previous values(0x3f and 0x0f) because only the unused bits are changed. I can use AVRISP mkII and USBasp to Burn Bootloader with both avrdude 6.0.1-arduino5 and 6.3.0-arduino2. I had previously thought this change was possible only because of a difference in avrdude 6.3 and that you'd have to add an avrdude tool to the MightyCore installation but this indicates that may not be the case and you can stick with the current system of just using whichever avrdude is currently installed for the Arduino AVR Boards version the user has.

Of course that's not a very thorough test, I've only really checked this out with the Arduino AVR Boards 1.6.12/avrdude 6.3.0-arduino2 since I wasn't sure which approach you'd want to take with this.

I tried applying the patch to your avrdude.conf and it doesn't want to work because it's an older version than the one from Arduino AVR Boards I patched.

[per1234]

It works with Arduino as ISP using avrdude 6.0.1-arduino5 and 6.3.0-arduino2 also. For some reason my USBtinyISP has been failing verification of the bootloader lately so I wasn't able to test beyond that step with that programmer. It works intermittently with 512 byte bootloaders but anything larger almost always fails. I have a couple new ones on their slow way here from China.

[MCUdude]

Great! I'm about to apply the patch! I'll be back with more info soon

[MCUdude]

I'm quite sure I didn't succeed applying the patch. After following your instructions this is the output of the patch command

$ patch --input=no-lock-byte-read-mask.patch --binary --output=avrdude.conf
patching file ChangeLog
Hunk #1 FAILED at 1.
1 out of 1 hunk FAILED -- saving rejects to file avrdude.conf.rej
patching file NEWS
Hunk #1 FAILED at 9.
1 out of 1 hunk FAILED -- saving rejects to file avrdude.conf.rej
patching file avrdude.conf.in
Hunk #1 FAILED at 1819.
Hunk #2 succeeded at 2424 with fuzz 1 (offset -902 lines).
Hunk #3 FAILED at 2607.
Hunk #4 FAILED at 2799.
Hunk #5 FAILED at 2991.
Hunk #6 FAILED at 3183.
Hunk #7 FAILED at 3340.
Hunk #8 FAILED at 3542.
Hunk #9 FAILED at 3750.
Hunk #10 FAILED at 3958.
Hunk #11 FAILED at 4152.
Hunk #12 FAILED at 4385.
Hunk #13 FAILED at 4547.
Hunk #14 FAILED at 4729.
Hunk #15 FAILED at 4914.
Hunk #16 FAILED at 5140.
Hunk #17 FAILED at 5331.
Hunk #18 FAILED at 5477.
Hunk #19 FAILED at 5630.
Hunk #20 FAILED at 5787.
Hunk #21 FAILED at 5947.
Hunk #22 FAILED at 6925.
Hunk #23 FAILED at 7137.
Hunk #24 FAILED at 7351.
Hunk #25 FAILED at 7563.
Hunk #26 FAILED at 7753.
Hunk #27 FAILED at 7942.
Hunk #28 FAILED at 8128.
Hunk #29 FAILED at 8310.
Hunk #30 FAILED at 9177.
Hunk #31 FAILED at 9367.
Hunk #32 FAILED at 9575.
Hunk #33 FAILED at 10746.
Hunk #34 FAILED at 10937.
Hunk #35 FAILED at 11141.
Hunk #36 FAILED at 11338.
Hunk #37 FAILED at 11525.
Hunk #38 FAILED at 11714.
Hunk #39 FAILED at 11902.
Hunk #40 FAILED at 12090.
Hunk #41 FAILED at 12244.
Hunk #42 FAILED at 12435.
Hunk #43 FAILED at 13859.
42 out of 43 hunks FAILED -- saving rejects to file avrdude.conf.rej

Any ideas? If yours succeeded, maybe you could upload it here so I can add that one instead?

[per1234]

I had the same problem. It's because your avrdude.conf is too old. The patch was successful on the Arduino AVR Boards 1.6.12 ardude.conf because it's more up to date. Here's the options I see:

1. You could use the patched file Arduino AVR Boards 1.6.12 ardude.conf and then merge back in your modifications. That file has the syntax incompatible with avrdude 6.0.1-arduino5 which will cause incompatibility with previous Arduino AVR Boards versions so you will also need to add an avrdude 6.3.0 tool to MightyCore. From cmaglie's recent statements it sounds like the current system of using Arduino AVR Boards tools is not recommended so that may be required anyway.
2. Manually copy the modified lock read masks from the patch to your current avrdude.conf which, from the test I described above, appears to retain compatibility with avrdude 6.0.1-arduino5.

I'm happy to help out with any of this but my ability to test is limited to the MCUs I have on hand(ATmega1284P, ATmega328P and ATmega168 are the ones relevant to your cores).

[MCUdude]

I think option 2 is the best choice. A lot of users are still using older versions of the IDE. I've very little (ok, no experience) with the patch command, how it works (identifying what to change and what to leave) and how it can be done manually. I'd really appreciate some help with this one 😃 After the patched file is added I should be able to do this on the other cores as well.

If I'm a bit late to reply is it because I'm on vacation, and doesn't got access to a computer all the time. I get all the notifications on my phone though 😉
Thanks for being such a resource for me and the rest of the community. It's people like you that make sure code and cores like this don't get obsolete because the Arduino team always seems to break backwards compatibility with every update.

[per1234]

Option 1 won't break backwards compatibility because avrdude 6.3.0-arduino2 will be installed with MightyCore regardless of the IDE version. The downside is that this means a larger install and duplicate tools. I'm hoping the Arduino developers will come up with a way that tools can be shared between cores without causing the sort of problems we're experiencing now but this will be tricky and any fix will only apply to future IDE versions. The MightyCore specific tools approach will be more of a hassle for manual installation because you will have to either include the tools in this repository to allow a installation via a single download or instruct the users to install the tools separately, adding to the complexity of the installation process.

This situation has definitely been a project since I have quite a few different 3rd party boards projects I'm involved with. I haven't done any work on some of them yet because I'm still trying to figure out the best approach to fixing all this breakage while maintaining backwards compatibility. Throw in Eclipse Arduino plugin compatibility, etc. and it gets even more complex.

[MCUdude]

The absolute best option would be to make sure all users runs the same version of avrdude. If the first option won't break backwards compatibility and force users to use "my" version of avrdude (in this case avrdude 6.3) then I think we should stick with this. I'm not a conservative person myself, and I think that new and better software is great! 👍 IIRC the avrdude executable
is less than a megabyte, so duplicated 'dudes is what I'd call "splitting hairs" when it comes to space. I'll agree it's not that elegant though..

When it comes to the tool dependencies I've decided to wait until the dust settles. There's a long going discussion, and dealing with the Eclipse add-on makes it even harder. Hopefully there will soon be a fix we all can agree on.