Library use of POWER_ON and RESET_N

[janakelarsson]

After some erratic behavior of the NB and finally a bricked device I looked in the system integration manual and found differences between the use of the POWER_ON and RESET_N pins in the manual and in the library.

(The circuitry on the board makes high output on the microcontroller result in a low level on the modem pins, I'll just describe what happens on the modem below.)

The library pulls down POWER_ON permanently, and uses RESET_N to reset the modem.

The modem expects a low pulse (length 150-3200 ms) on the POWER_ON pin for startup, and another pulse (length at least 1500 ms) for powerdown. It also advises against using the RESET_N pin at all.

I suspect I have bricked one device by using the library restart function too liberally, and that the erratic behavior was caused by an earlier unexpected forced restart using RESET_N at the wrong moment.

I have tried a rewritten variant of the library on a new device, following the SARA-R4 datasheet and systems integration manuals, and it seems to work. I would prefer to remove the use of RESET_N even from the arduino core used, in variant.cpp, I have used that in my tests too.

The systems integration manual recommends monitoring the V_INT pin to detect if the modem is on or not. Unfortunately, this is not connected to the microcontroller on the NB. I am considering options for that.

I can submit a pull request on the rewritten library. Should I?

[hdahle]

I've been struggling with exactly the same, and my conclusions are similar to @janakelarsson

ModemClass::begin(bool restart) contains code that can permanently destroy the SARA-410M module:

  if (_resetPin > -1 && restart) {
    pinMode(_resetPin, OUTPUT);
    digitalWrite(_resetPin, HIGH);
    delay(100);
    digitalWrite(_resetPin, LOW);

This generates a 100ms LOW pulse on the SARA RESET_N pin, which does not reset the device and can actually harm the device:
"An abrupt removal of the VCC supply or forcing a low level on the RESET_N input once the boot of
SARA-R4 series modules has been triggered may lead to an unrecoverable faulty state!" (page 23 in https://www.u-blox.com/sites/default/files/SARA-R4_SysIntegrManual_%28UBX-16029218%29.pdf)

ModemClass::end() also contains code that can permanently destroy the SARA-410M module:

void ModemClass::end()
{
  _uart->end();
  digitalWrite(_resetPin, HIGH);

  // power off module
  digitalWrite(_powerOnPin, LOW);
}

The problem is

digitalWrite(_resetPin, HIGH);

Driving the RESET_N pin HIGH from the processor results in a LOW input to the SARA module. This should never be done except in the case of emergency shutdown. The uBLOX system integration manual says:

"Forcing a low level on the RESET_N input during SARA-R4 series modules normal operations may lead to an unrecoverable faulty state" (page 24 in https://www.u-blox.com/sites/default/files/SARA-R4_SysIntegrManual_%28UBX-16029218%29.pdf)

This part of ModemClass::end() simply does not work at all, but has no fatal consequences:

  // power off module
  digitalWrite(_powerOnPin, LOW);

This is not quite how the SARA R410 should be powered off. As @janakelarsson points out, a power-off sequence is initiated by driving the SARA PWR_ON input low for a duration of 1.5-3.2 seconds. Hence, the _powerOnPin should be driven HIGH for 1.5-3.2s to power the SARA off.

A lot of people probably have bricked their Arduino MKRNB1500 devices simply by using the Modem library. It would be great to update the library!

[janakelarsson]

The library differences can be seen at master...janakelarsson:powerpulse

[hdahle]

@janakelarsson - did you also take a look at variant.h / variant.cpp? I am not sure when initVariant() is called, but the same issues apply:

initVariant() in variant.cpp attempts to power off the module by doing digitalWrite(SARA_PWR_ON, LOW); which does not power off the module at all.

initVariant() also uses digitalWrite(SARA_RESETN, HIGH); in order to put SARA modem in reset on start to conserve power. As we know, using the RESETN signal this way does not reset the device, nor does it conserve power. It merely risks putting the device in an unrecoverable fault state.

void initVariant() {
#if defined(USE_BQ24195L_PMIC)
  pinMode(ADC_BATTERY, OUTPUT);
  digitalWrite(ADC_BATTERY, LOW);
  delay(10);
  pinMode(ADC_BATTERY, INPUT);
  delay(100);

  bool batteryPresent = analogRead(ADC_BATTERY) > 600;
  if (batteryPresent) {
    enable_battery_charging();
  }
  disable_battery_fet(!batteryPresent);
  set_voltage_current_thresholds();
#endif

  // power off the module
  pinMode(SARA_PWR_ON, OUTPUT);
  digitalWrite(SARA_PWR_ON, LOW);

  // put SARA modem in reset on start to conserve power if it's not used
  pinMode(SARA_RESETN, OUTPUT);
  digitalWrite(SARA_RESETN, HIGH);

  // set RTS to LOW
  pinMode(SARA_RTS, OUTPUT);
  digitalWrite(SARA_RTS, LOW);
}

[janakelarsson]

Yes, see arduino/ArduinoCore-samd@master...janakelarsson:powerpulse, the code makes no attempt to power down the module on reboot, to handle that properly we'd need to monitor V_INT, which the hardware does not allow just now.

[aentinger]

Should be fixed by #53 and arduino/ArduinoCore-samd#549 ... maybe you can take a look @janakelarsson ?

[janakelarsson]

No it does not.

The problem is that the power-on and power-off functions in the library does not follow the data sheet. At all. Can I make this clearer? The power-on procedure of the modem is not followed in the library. The power-off procedure of the modem is not followed in the library. It is more luck than anything else that the modem starts at all.

I've rewritten the begin() and end() to follow the data sheet power-on and power-off procedure, including a possibility for AT power-off by a new method shutdown().

The change in #53 only attempts to hide the problem by setting the default begin() argument restart=false. That would avoid the immediate threat of an unwary user bricking the modem by a simple call to begin(), but does not fix the issue. A more savvy user would expect restart=true to restart the modem, whereas there is a real risk that the modem is bricked by issuing begin(true).

The code change in arduino/ArduinoCore-samd#549 is indeed the same as the one I proposed for the variant core, but I would rather that the comment is changed too, see arduino/ArduinoCore-samd@master...janakelarsson:powerpulse Otherwise, the comment will only confuse the reader.

Edited to add: of course, the intended effects of #53 (add file system) and arduino/ArduinoCore-samd#549 (add second stage bootloader) are certainly good (but they do not have the side effect to fix this problem).

[aentinger]

@giulcioffi 👋 ☕ Can you please take another look at this?

[janakelarsson]

See page 23 onwards in https://www.u-blox.com/sites/default/files/SARA-R4_SysIntegrManual_%28UBX-16029218%29.pdf

[zbelding]

This issue was raised December 2019: https://forum.arduino.cc/index.php?topic=654725.0

Also see from ublox SARA R4 data sheet:

[zbelding]

It is more luck than anything else that the modem starts at all.

I agree.

[ircole]

I'm fairly new to the Arduino world so forgive me asking if this isn't proper. I have this problem (and more) with the MKRNB1500. When do these issues that have been identified picked up in an update to the library? From my observations it doesn't appear that Arduino is paying much attention to the MKR1500 which is disappointing as my plan was to use it for monitoring crude and salt water levels in storage tanks as well as monitor several other sensors. My experience is that it is not reliable enough to depend upon for 24/7 365. I was excited when I found the MKR1500 but have spent many hours trying to get it to not hang up. The power off and on is one of the areas. How timeouts are handled is another. My intent isn't to throw stones, rather understand how support for Arduino devices works. Maybe the intent isn't to have something that can be depended upon rather just for experimenting with.

[janakelarsson]

I believe this was in the library from the start. From what I understand, the MKR NB 1500 library was started from the MKR GSM 1400 library which uses the UBLOX SARA-U201 modem. The NB 1500 uses a UBLOX SARA-R410M modem, which has a different power-on procedure from the SARA-U201.

I have not used the MKR GSM 1400 so I cannot say how well the MKRGSM library works. Perhaps there are no problems.

However, it seems to me that the power-on sequence of the MKRNB library follows the system integration manual of the SARA-U201 instead of the system integration manual of the SARA-R410M. Of course, since the MKR NB 1500 uses a SARA-R410M, this is bad. The MKRNB library should follow the system integration manual of the SARA-R410M modem.

The changes I propose makes the MKRNB library follow the system integration manual of the SARA-R410M modem, as far as the hardware allows. I would have liked to be able to monitor the V_INT pin of the modem, as the system integration manual recommends, but that is not possible with the current hardware.

[zbelding]

I would prefer to remove the use of RESET_N even from the arduino core used, in variant.cpp

I can tell you from my experience with this modem, there are certain situations where the modem will become unresponsive to AT commands(even with proper command response timeouts which can be up to 2 minutes). Weak cell signal areas seem to be the main culprit here.

The only way to recover from this is a hard reset of the modem. I realize this is not recommended by ublox. Hopefully ublox will have another firmware update soon to address this.

Rather than completely removing RESET_N functionality, I think it would be better to only reset the modem after so many(example 20) response timeouts. This only works if every AT command in the library uses the recommend response timeouts listed in the AT command manual.

[janakelarsson]

(SMS command timeout is 3 minutes according to the manual)

There are four ways to shut down the modem:

1. AT+PWROFF which was earlier used in the NB.shutdown() method but after the patch is also available directly as MODEM.shutdown(). As you say this will only work if the AT interface is responding.
2. A proper shutdown pulse on PWR_ON, this was never available in the library before my patch. I do not know if this needs a working AT interface, since I have not tested this. Perhaps you have tested this directly, bypassing the library?
3. Using RESET_N to do EMERGENCY power-off. I would rather have a separate method for that, something like .hardReset()
4. Removing power to VCC.

There is currently no way to check for functionality of the PWR_ON pulse, other than the AT interface becoming unresponsive. One could, in principle, wait until .noop() times out and then assume that the modem is shutdown but that is all.

There is also no way to predict the shutdown time of the modem, the system integration manual gives no guide or timeout. An "OK" response for the "AT+PWROFF" command does not mean the modem has shut down, it means that the command has been received and understood. Best would be if the ability to measure V_INT was added to the hardware. I have contemplated doing this on my own unit.

[zbelding]

2\. A proper shutdown pulse on PWR_ON, this was never available in the library before my patch. I do not know if this needs a working AT interface, since I have not tested this.

I will have to try this out when the modem is unresponsive. I have a suspicion that it may not work when the AT terminal is unresponsive, but I will try it. this is from the integration manual:

RESET_N line should be set low only if reset or shutdown via AT commands fails or if the module does not reply to a specific AT command after a time period longer than the one defined in SARA-R4 series AT commands manual[2].

They don't mention shutdown via PWR_ON pin pulse when module does not reply...

[aentinger]

Hi @janakelarsson 👋
Given that #55 and arduino/ArduinoCore-samd#549 are merged do you consider this issue done?

[janakelarsson]

Library issue is handled as far as I am concerned. @zbelding may want to test, but perhaps it is better (s)he opens a new issue if there are problems.

However, arduino/ArduinoCore-samd#549 did not change the comment, which is now not just wrong wrt data sheet but also wrong wrt attempted behavior. I'll do a pull request for that.

[aentinger]

Thank you, please mention me at your ArduinoCore-samd PR and I shall merge it quickly.

[ClausElmann]

Hi @janakelarsson

My program ran for 1 aprox 1 month, before hanging in

boolean connected = false;
  while (!connected)
  {
    Serial.println("while (!connected)");
    if ((nbAccess.begin(PINNUMBER, APN) == NB_READY) &&
        (gprs.attachGPRS() == GPRS_READY))
    {
      Serial.println("GPRS_READY");
      connected = true;
    }
    else
    {
      Serial.println("Not connected");
      delay(1000);
    }
  }

And burning a new program didn't help, but disconnection the power worked.

Is there any method to do a modem reset from code?

Any help is highly appreciated :-)

[janakelarsson]

You could try the hardReset() method in the library, defined as follows:

void ModemClass::hardReset()
{
  // Hardware pin reset, only use in EMERGENCY
  digitalWrite(_resetPin, HIGH);
  delay(1000); // Datasheet says nothing, so guess we wait one second
  digitalWrite(_resetPin, LOW);
  setVIntPin(SARA_VINT_OFF);
}

If this does not work, I would guess power cycling is the only way.

I have bricked one NB1500 and finally gave up on the new one, could never get that to work stably. I moved to a SIMCom modem.

[ClausElmann]

Hi @janakelarsson

Thanks for getting back.

I'm in the early stages of a project and have only run tests on a board that has frozen twice in two months. I'm thinking it might be better to switch hardware now if the MKR NB 1500 won't work stably without the customer having to power off and reset the modem.

Can you guide me in the right direction for other hardware options?

Requirements:
NB-IoT and a similar model with WIFI (preferred)
Modbus shield
Connection of 2 temperature sensors/shield
2 relay outputs/shield

Can you recommend something?

I hope to be able to reuse some of the code I wrote for the MKR NB 1500.

[janakelarsson]

I can't give advice on that, sorry. I used the 2G-only SIM800L and will need to replace that soon when the 2G network closes down. I'm looking at other solutions myself.

[ClausElmann]

Hi @janakelarsson,

Thanks for your response. I recently discovered that NB-IOT sims are quite affordable (1 EUR/month) and have excellent coverage in Denmark, which is why I decided to go with the MKR NB 1500.

[lorithai]

You could try the hardReset() method in the library, defined as follows:

void ModemClass::hardReset()
{
  // Hardware pin reset, only use in EMERGENCY
  digitalWrite(_resetPin, HIGH);
  delay(1000); // Datasheet says nothing, so guess we wait one second
  digitalWrite(_resetPin, LOW);
  setVIntPin(SARA_VINT_OFF);
}

If this does not work, I would guess power cycling is the only way.

I have bricked one NB1500 and finally gave up on the new one, could never get that to work stably. I moved to a SIMCom modem.

Hard reset command is wrong. Time for high is 10s not 1s according to the datasheet. Not sure what 1s would do.
https://content.u-blox.com/sites/default/files/SARA-R4_DataSheet_UBX-16024152.pdf
page 35.

[janakelarsson]

Then do change that. As you may see in the code if you read the comments, that was not present in the manual when the code was written. I have not used a MKR NB for many years now.

Also, BEWARE, using hardReset may brick your device. I repeat: MAY BRICK YOUR DEVICE.

I'll unsubscribe to this issue now since it is of no interest to me anymore.

[lorithai]

Oh, was not aware the documentation didn't specify this at that time.

Yeah, aware that hardReset may brick the device, but as is, it is the only thing i have found that solves the freezing issue i am facing. It may brick the modem, but if i don't use it the entire device is already pretty much a brick.

I did find one interesting parameter which affects the frequency of modem freezes for me. The more data that is sent, the more frequent the freezes.

[ClausElmann]

Oh, was not aware the documentation didn't specify this at that time.

Yeah, aware that hardReset may brick the device, but as is, it is the only thing i have found that solves the freezing issue i am facing. It may brick the modem, but if i don't use it the entire device is already pretty much a brick.

I did find one interesting parameter which affects the frequency of modem freezes for me. The more data that is sent, the more frequent the freezes.

@lorithai

One potential solution could be to add a normally closed switch in-line with the power supply to the device. This would allow you to power cycle the device by briefly opening and then closing the switch, which could help reset the modem and potentially resolve any issues causing it to stop working.

To control the switch from your program, you could use a digital output pin on the Arduino to control a transistor or relay that switches the power to the device on and off.

I did't try it out, but it may work?

[Bild4u]

Similar struggles here with the NB1500. I too am thinking of power cycling the NB1500 (it'll be a while before I get around to testing though). I'd be very thankful for any suggestions regarding alternative hardware options (getting away from the NB1500). Cheers.

[TomWesthoff]

I'm a bit confused, is the MKRNB library fixed now to follow the correct reset and power down as discussed above? I need to be able to reset/reboot/powerdown the modem when it gets stuck. Is it safe to call these functions now? And for the somewhat inexperienced, like myself, what are the names of these (hopefully fixed) functions?

[KonRoz]

@TomWesthoff Were you able find a solution to the reset problem?

[TomWesthoff]

No. I as afraid to try. Not sure of the best way to do it. I would like to see sample code that is done correctly using the newest MKRNB library.