Nuki 2.0 - Speed up bluetooth initial connection

danergo · June 15, 2020, 9:39am

Hi,

I’m trying to interface with Nuki 2.0 Smart Lock with native C++ under Linux.
I’ve got a lot of things working now (subscripting to USDIO, encrypting data, sending it, getting indications, etc), but one thing I’d like to improve:

The initial connection to the SL takes too much time. This case can be easily validated with standard commands:

gatttool -b MAC -I

Then type “connect”. It will (probably) time out, and you have to retry for random times until it succeeds.
Sometimes it even need 15-20 retrials (which is a lot in time).

I know it’s a battery operated device, but the phone app seems to find it faster.

I’m curious of the small detail about when is the right moment to initiate the connection? Like in X milliseconds after the device sent an iBeacon? Or Y milliseconds after the iBeacon?
Or try as fast as the system able until succeed? Or do something before initiate the connection?

Do you have some advice for me (just like for the RSSI indicator’s showing an updated state of the SL)?

Thanks.

danergo · June 18, 2020, 11:51am

For example, when I use the keypad, it seems very fast. I don’t have gateway, only KT and Keypad.

So I think there must be some clever way to make the connection faster, but it might be not public.

Guys please confirm this, in either way. I.e.: keypad uses proprietary (fast) connection method; or keypad uses connection method of this-and-that which is (not)applicable for the current API (becuse this-and-that|in the following way).

Thank you

Juergen · June 18, 2020, 2:50pm

The Keypad uses the same BLE connection mechanism as the bridge and smartphones uses.
Please make sure that you use the latest Smart Lock firmware and that you don’t have connections open from other devices. The Smart Lock supports multiple connections only in recent firmware versions and even then the number of concurrent connections is limited (max. 4).

danergo · June 19, 2020, 8:42am

Thanks Jürgen!

I believe my KeyTurner is up to date (FW: 2.7.20).
I have only one KeyTurner and one Keypad. The KeyTurner is registered for 2 phones.

When I try to connect to the KeyTurner with gatttool, both phones are idle (screen off, Nuki app not running). Maybe Nuki app tries to connect even during the phone sleeping? I don’t think so.

Here is a log from btmon when I try to connect with gatttool:

# btmon
Bluetooth monitor ver 5.50
= Note: Linux version 4.19.97-v7l+ (armv7l)                            0.742019
= Note: Bluetooth subsystem version 2.22                               0.742027
= New Index: AA:BB:CC:DD:EE:FF (Primary,UART,hci0)              [hci0] 0.742030
= Open Index: AA:BB:CC:DD:EE:FF                                 [hci0] 0.742033
= Index Info: AA:BB:CC:D.. (Cypress Semiconductor Corporation)  [hci0] 0.742035
@ MGMT Open: bluetoothd (privileged) version 1.14             {0x0001} 0.742038
@ MGMT Open: btmon (privileged) version 1.14                  {0x0002} 0.742321
< HCI Command: LE Set Scan Parameters (0x08|0x000b) plen 7   #1 [hci0] 4.737267
        Type: Passive (0x00)
        Interval: 60.000 msec (0x0060)
        Window: 30.000 msec (0x0030)
        Own address type: Public (0x00)
        Filter policy: Ignore not in white list (0x01)
> HCI Event: Command Complete (0x0e) plen 4                  #2 [hci0] 4.737714
      LE Set Scan Parameters (0x08|0x000b) ncmd 1
        Status: Success (0x00)
< HCI Command: LE Set Scan Enable (0x08|0x000c) plen 2       #3 [hci0] 4.737767
        Scanning: Enabled (0x01)
        Filter duplicates: Enabled (0x01)
> HCI Event: Command Complete (0x0e) plen 4                  #4 [hci0] 4.738160
      LE Set Scan Enable (0x08|0x000c) ncmd 1
        Status: Success (0x00)
> HCI Event: LE Meta Event (0x3e) plen 42                    #5 [hci0] 6.099681
      LE Advertising Report (0x02)
        Num reports: 1
        Event type: Connectable undirected - ADV_IND (0x00)
        Address type: Public (0x00)
        Address: NU:KI:KT:MA:CA:DR (Nuki Home Solutions GmbH)
        Data length: 30
        Flags: 0x06
          LE General Discoverable Mode
          BR/EDR Not Supported
        Company: Apple, Inc. (76)
          Type: iBeacon (2)
          UUID: 669a0c20-0008-6c91-e411-015500e22ea9
          Version: 48661.62728
          TX power: -59 dB
        RSSI: -78 dBm (0xb2)
< HCI Command: LE Set Scan Enable (0x08|0x000c) plen 2       #6 [hci0] 6.099747
        Scanning: Disabled (0x00)
        Filter duplicates: Disabled (0x00)
> HCI Event: Command Complete (0x0e) plen 4                  #7 [hci0] 6.101862
      LE Set Scan Enable (0x08|0x000c) ncmd 1
        Status: Success (0x00)
< HCI Command: LE Create Connection (0x08|0x000d) plen 25    #8 [hci0] 6.101916
        Scan interval: 60.000 msec (0x0060)
        Scan window: 60.000 msec (0x0060)
        Filter policy: White list is not used (0x00)
        Peer address type: Public (0x00)
        Peer address: NU:KI:KT:MA:CA:DR (Nuki Home Solutions GmbH)
        Own address type: Public (0x00)
        Min connection interval: 30.00 msec (0x0018)
        Max connection interval: 50.00 msec (0x0028)
        Connection latency: 0 (0x0000)
        Supervision timeout: 420 msec (0x002a)
        Min connection length: 0.000 msec (0x0000)
        Max connection length: 0.000 msec (0x0000)
> HCI Event: Command Status (0x0f) plen 4                    #9 [hci0] 6.102446
      LE Create Connection (0x08|0x000d) ncmd 1
        Status: Success (0x00)
> HCI Event: LE Meta Event (0x3e) plen 19                   #10 [hci0] 7.476997
      LE Connection Complete (0x01)
        Status: Success (0x00)
        Handle: 64
        Role: Master (0x00)
        Peer address type: Public (0x00)
        Peer address: NU:KI:KT:MA:CA:DR (Nuki Home Solutions GmbH)
        Connection interval: 48.75 msec (0x0027)
        Connection latency: 0 (0x0000)
        Supervision timeout: 420 msec (0x002a)
        Master clock accuracy: 0x00
@ MGMT Event: Device Connected (0x000b) plen 43        {0x0002} [hci0] 7.477047
        LE Address: NU:KI:KT:MA:CA:DR (Nuki Home Solutions GmbH)
        Flags: 0x00000000
        Data length: 30
        Flags: 0x06
          LE General Discoverable Mode
          BR/EDR Not Supported
        Company: Apple, Inc. (76)
          Type: iBeacon (2)
          UUID: 669a0c20-0008-6c91-e411-015500e22ea9
          Version: 48661.62728
          TX power: -59 dB
@ MGMT Event: Device Connected (0x000b) plen 43        {0x0001} [hci0] 7.477047
        LE Address: NU:KI:KT:MA:CA:DR (Nuki Home Solutions GmbH)
        Flags: 0x00000000
        Data length: 30
        Flags: 0x06
          LE General Discoverable Mode
          BR/EDR Not Supported
        Company: Apple, Inc. (76)
          Type: iBeacon (2)
          UUID: 669a0c20-0008-6c91-e411-015500e22ea9
          Version: 48661.62728
          TX power: -59 dB
< HCI Command: LE Read Remote Used... (0x08|0x0016) plen 2  #11 [hci0] 7.477210
        Handle: 64
> HCI Event: Command Status (0x0f) plen 4                   #12 [hci0] 7.479342
      LE Read Remote Used Features (0x08|0x0016) ncmd 1
        Status: Success (0x00)
> HCI Event: Command Complete (0x0e) plen 14                #13 [hci0] 7.479357
      LE Read Remote Used Features (0x08|0x0016) ncmd 1
        Status: Success (0x00)
        00 00 00 00 00 00 00 00 00 00                    ..........      
> HCI Event: LE Meta Event (0x3e) plen 12                   #14 [hci0] 7.993969
      LE Read Remote Used Features (0x04)
        Status: Connection Timeout (0x08)
        Handle: 64
        Features: 0x2d 0x00 0x00 0x00 0x00 0x00 0x00 0x00
          LE Encryption
          Extended Reject Indication
          Slave-initiated Features Exchange
          LE Data Packet Length Extension
> HCI Event: Disconnect Complete (0x05) plen 4              #15 [hci0] 7.994591
        Status: Success (0x00)
        Handle: 64
        Reason: Connection Timeout (0x08)
@ MGMT Event: Device Disconnected (0x000c) plen 8      {0x0002} [hci0] 8.027693
        LE Address: NU:KI:KT:MA:CA:DR (Nuki Home Solutions GmbH)
        Reason: Connection timeout (0x01)
@ MGMT Event: Device Disconnected (0x000c) plen 8      {0x0001} [hci0] 8.027693
        LE Address: NU:KI:KT:MA:CA:DR (Nuki Home Solutions GmbH)
        Reason: Connection timeout (0x01)

Even though the device seems “Connected”, “LE Read Remote Used Features (0x04)” seems timing out ~500ms after the request.

Maybe I would need to tune this to a bit larger somehow, or disable querying Used Features?

danergo · June 20, 2020, 5:24am

I got new information on this. Timing out of “Read Remote Used Features” is exactly what it says. If I would disable the Remote Used Features query function, then the next communication trial would fail/timeout.

The interesting thing here is that my device is an RPi4’s internal bluetooth, and it can connect to at least dozens of devices without timeout. Only Nuki times out, but I don’t know why.

danergo · June 21, 2020, 3:27pm

Hi, is there anything I can tune on a standard Linux BT (hci) device for making the connection to Nuki more fast? I just did some sniffs: the phone uses a supervision timeout of 625ms, but my RPi uses 52.5ms.

Is this something can prevent RPi to connect? (It can connect to anything else!)
Although I have changed this value to 625 for the RPi, it does not seem any faster. Timeouts a lot, I need to wait minutes for a single successful connection.

I thought of the Pi’s low quality cypress BT solution, but it is really close to the Nuki (3m - no walls, direct sight). I’d appreciate some help form you guys!

The Cypress chip in Pi is only BT4.2 (although they hacked it through the standards of 5.0), so some features are missing. I’m pretty sure that my phone is a real 5.0 device. What about Nuki2.0? Is it also BTLE5.0? Or older?

With my sniffer I can see a lot of MTU request packets from the Nuki, but on the RPi it seems it’s waiting for the answer for “LE Read Remote Used Features” - and it keeps waiting for it until timeout. Is it possible that Nuki changes frequency during connection?

Thanks.

danergo · June 21, 2020, 7:38pm

One more thing I have noticed:

When my phone connects to Niki, it first scans Nukis beacons (scan req+scan rsp).
Is there any extra info in the beacons that coordinates the connection request?

Thanks.

danergo · June 23, 2020, 8:05am

Noone knows the answer for this?

First I thought RPi’s BT is wrong/miss packets. But then what is the reason for this:
once RPi finally can connect, no bytes are lost and Pi can communicate fluently with Nuki afterwards.

Juergen · June 23, 2020, 11:01am

Tested it with a raspberry & Smart Lock 2.0 and Opener and works for me as it should. No connection problems.

You can also try it with the “nrf connect” app on your mobile phone.

If that works without issues the problem is most likely your raspberry. The only things that you can change on the Smart Lock that have an impact on BLE is the Battery saving (“Fast” will use shorter advertising intervals) and switching off HomeKit.

danergo · June 23, 2020, 11:36am

Thanks Jürgen for your efforts. My Battery saving setting is set to Automatic (Default) on the SmartLock.

I now tried to put RPi4 very close to the lock (1.5m), and still the same issue.
Also tried on the phone with nRF connect it works (but the phone app works 100% anyway).

So it seems my RPi is not OK (Although other devices can work with it). Can you please tell me how exactly did you tried with RPi?
Which distribution and which commands?

Mine here:

root@rpi:~# gatttool -t random -b OTHERWORKINGDEVICE1 --characteristics
handle = 0x0002, char properties = 0x0a, char value handle = 0x0003, uuid = 00002a00-0000-1000-8000-00805f9b34fb
handle = 0x0004, char properties = 0x02, char value handle = 0x0005, uuid = 00002a01-0000-1000-8000-00805f9b34fb
handle = 0x0006, char properties = 0x02, char value handle = 0x0007, uuid = 00002a04-0000-1000-8000-00805f9b34fb
handle = 0x0008, char properties = 0x02, char value handle = 0x0009, uuid = 00002aa6-0000-1000-8000-00805f9b34fb
handle = 0x000b, char properties = 0x20, char value handle = 0x000c, uuid = 00002a05-0000-1000-8000-00805f9b34fb
handle = 0x000f, char properties = 0x0c, char value handle = 0x0010, uuid = 99fa0002-338a-1024-8a49-009c0215f78a
handle = 0x0011, char properties = 0x12, char value handle = 0x0012, uuid = 99fa0003-338a-1024-8a49-009c0215f78a
handle = 0x0015, char properties = 0x1e, char value handle = 0x0016, uuid = 99fa0011-338a-1024-8a49-009c0215f78a
handle = 0x0019, char properties = 0x12, char value handle = 0x001a, uuid = 99fa0021-338a-1024-8a49-009c0215f78a
handle = 0x001c, char properties = 0x02, char value handle = 0x001d, uuid = 99fa0029-338a-1024-8a49-009c0215f78a
handle = 0x001e, char properties = 0x02, char value handle = 0x001f, uuid = 99fa002a-338a-1024-8a49-009c0215f78a
handle = 0x0021, char properties = 0x0c, char value handle = 0x0022, uuid = 99fa0031-338a-1024-8a49-009c0215f78a
root@rpi:~#
root@rpi:~# gatttool -b OTHERWORKINGDEVICE2 --characteristics
handle = 0x0002, char properties = 0x20, char value handle = 0x0003, uuid = 00002a05-0000-1000-8000-00805f9b34fb
handle = 0x0006, char properties = 0x4e, char value handle = 0x0007, uuid = 00002a00-0000-1000-8000-00805f9b34fb
handle = 0x0008, char properties = 0x4e, char value handle = 0x0009, uuid = 00002a01-0000-1000-8000-00805f9b34fb
handle = 0x000a, char properties = 0x02, char value handle = 0x000b, uuid = 00002a04-0000-1000-8000-00805f9b34fb
handle = 0x000d, char properties = 0x16, char value handle = 0x000e, uuid = 0000fe51-0000-1000-8000-00805f9b34fb
handle = 0x0010, char properties = 0x08, char value handle = 0x0011, uuid = 0000fe52-0000-1000-8000-00805f9b34fb
root@rpi:~#
root@rpi:~# gatttool -b NUKI --characteristics
connect error: Connection timed out (110)
root@rpi:~#

But sometimes works. Sometimes I have to retry for 60 times(!).

Thanks a lot!

Juergen · June 23, 2020, 1:18pm

Raspberry Pi 3 Model B Rev 1.2
PRETTY_NAME=“Raspbian GNU/Linux 8 (jessie)”
NAME=“Raspbian GNU/Linux”
VERSION_ID=“8”
VERSION=“8 (jessie)”

Tools: btmon and gatttool (same commands that you used)

danergo · June 23, 2020, 1:42pm

Thanks!

Mine is:
RPi4 Model B 1.1 (1GB RAM), Q2 2019 (Mfg by Sony)
PRETTY_NAME=“Raspbian GNU/Linux 10 (buster)”
NAME=“Raspbian GNU/Linux”
VERSION_ID=“10”
VERSION=“10 (buster)”

From my earlier experience, this RPi4 has the exact same Cypress BT+Wifi chip as Pi3.

giejay · June 24, 2020, 9:25pm

I have a raspberry pi zero and a raspberry pi 2, both connected to a Smart Lock. I’m using Python to unlock/lock the Nukis. Takes about 10 seconds in total. If you need help in testing some commands, I would be glad to.

danergo · June 25, 2020, 5:43am

@giejay: nice, thank you.

First, can you share what kind of Bluetooth do you use with PI2 and PI0?
(Or you have PI0W?)

Thank you!

giejay · July 3, 2020, 1:44pm

Hi,

Im using the following repo: https://github.com/giejay/nukiPyBridge

On a raspberry pi 3 and a zero W. Im not exactly sure what you mean with what kind of bluetooth. It just the built in bluetooth module of both pi’s.

I thought it was the pi 2, but of course, that one does not have BT built in.

danergo · July 4, 2020, 5:41am

Thanks, yes understand.

nukiPyBridge is using gatttool for its backend, and implemented a retrial mechanism in case it’s not able to connect.

10s anyway is not bad.

giejay · July 5, 2020, 9:59am

Yes I know, I implemented the retry mechanism after forking it;)

I agree 10 sec is not bad, but nowhere near the speed of the app so I would also like to speed up my code even more.

danergo · July 5, 2020, 10:27am

I have the information that Nuki’s upcoming 2.8 beta will contain a possible fix which probably will solve this tiny incompatibility with RPis.

danergo · November 23, 2020, 7:16pm

@Juergen: Do you have any of you an RPi4 near your hands? Cypress had finally heard our voice and fixed something on their side (RPi4 has a Cypress BT+Wifi module):

It would be good if you can test it, because I don’t have my SL2.0 with me

Juergen · November 24, 2020, 2:52pm

No, sorry.