Does not connect to new Flex hardware (e.g., Wattpilot Home FLEX 11 C6)

[lunchpack]

The python module is not able to connect to the current FLEX hardware (which was released just few weeks ago).
Error message is that the password is wrong / authentication failed.

It correctly identifies the serial number, though (obfuscated below for privacy).

I am 100% sure i exported the correct password. For sake of clarity and testing I also exported the system set default password and the Vcode, neither of them worked.

(.venv) @Mac ~ % wattpilotshell
INFO:wattpilot:Wattpilot None initialized
INFO:wattpilot.wattpilotshell:Automatically connecting to Wattpilot ...
INFO:wattpilot:Wattpilot connected
INFO:websocket:Websocket connected
INFO:wattpilot:Connected to WattPilot Serial <xxx…>
ERROR:wattpilot:Authentication failed: Wrong password

Best regards
Christian

[peterpoetzi]

Hi, you need to have a look at the hash attibute of the authRequired message. It can currently have the value pbkdf2, bcrypt or no value (which also means pbkdf2 for non wattpilot-flex chargers).

(Update: I discovered that very early wattpilot_flex devices do not send the hash attibute, here wattpilot_flex must be hardcoded to always use bcrypt)

So the decision tree could look like this:

• if hash present, use the advertised algorithm
• else if "devicetype":"wattpilot_flex" is set, use bcrypt
• else use pbkdf2

For a better performance, newer devices are set to bcrypt instead of pbkdf2.

In javascript it could look like this:

function sha256(a){
    return CryptoJS.SHA256(a).toString();
}
async function bcrypt(password, serial, iterations = 8) {
    password = sha256(password);
    let bcrypt = dcodeIO.bcrypt; // https://github.com/dcodeIO/bcrypt.js
    let salt = [];
    salt.push("$2a$");
    if (iterations < 10)
        salt.push("0");
    salt.push(iterations.toString());
    salt.push('$');
    serial = serial.padStart(16, '\0');
    salt.push(bcrypt.encodeBase64(serial, 16));
    salt=salt.join('');
    let hash = bcrypt.hashSync(password, salt);
    hash = hash.substring(salt.length);
    return hash;
}

[lunchpack]

Thank you @peterpoetzi ! Unfortunately I am not able to implement it. Maybe @joscha82 can have a look for the new gen hardware?

[Cellgalvano]

Hi!

If there is some version to test I will happily try it out! I got a Wattpilot Flex Home 22kW here. Moved from a basic Wattpilot 11kW.

BR

[dominiktopp]

Hi @peterpoetzi ,

Unfortunately I don't get what you mean.

Here is a part of my debug log. Some infos are x-ed, I don't know what of those numbers/string are secrets.

INFO:wattpilot.wattpilotshell:Automatically connecting to Wattpilot ...
DEBUG:wattpilot:Calling event handler for event type 'Event.WP_CONNECT ...
INFO:wattpilot:Wattpilot connected
INFO:websocket:Websocket connected
DEBUG:wattpilot:Calling event handler for event type 'Event.WS_OPEN ...
DEBUG:wattpilot.wattpilotshell:wp_handle_events(event={'type': <Event.WS_OPEN: (18,)>, 'wp': <wattpilot.Wattpilot object at 0x0000xxxxxxxxC050>},(<websocket._app.WebSocketApp object at 0x0000xxxxxxxxC2F0>,))
DEBUG:wattpilot.wattpilotshell:wp_handle_events(): MQTT client not yet initialized - status publishing skipped.
DEBUG:wattpilot:Message received: {"type":"hello","serial":"9xxxxxx6","hostname":"Wattpilot-9xxxxxx6","friendly_name":"Wattpilot","manufacturer":"fronius","devicefamily":"wattpilot","devicetype":"wattpilot_flex","devicesubtype":"wattpilot_flex_c6","version":"42.7","protocol":2,"proto":3,"secured":true}
DEBUG:wattpilot:Calling event handler for event type 'Event.WS_MESSAGE ...
INFO:wattpilot:Connected to WattPilot Serial 9xxxxxx6
DEBUG:wattpilot:Calling event handler for event type 'Event.WP_HELLO ...
DEBUG:wattpilot:Message received: {"type":"authRequired","token1":"17khxxxxxxxxxxxxxxxxxxxxxxxxW5LT","token2":"OOvuxxxxxxxxxxxxxxxxxxxxxxxx12v0"}
DEBUG:wattpilot:Calling event handler for event type 'Event.WS_MESSAGE ...
DEBUG:wattpilot:Message send: {"type": "auth", "token3": "33dbxxxxxxxxxxxxxxxxxxxxxxxxa5d4", "hash": "01bexxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx6583"}
DEBUG:wattpilot:Calling event handler for event type 'Event.WP_AUTH ...
DEBUG:wattpilot:Message received: {"type":"authError","message":"Wrong password"}
DEBUG:wattpilot:Calling event handler for event type 'Event.WS_MESSAGE ...
ERROR:wattpilot:Authentication failed: Wrong password
DEBUG:wattpilot:Calling event handler for event type 'Event.WP_AUTH_ERROR ...
DEBUG:wattpilot:Calling event handler for event type 'Event.WS_CLOSE ...
DEBUG:wattpilot.wattpilotshell:wp_handle_events(event={'type': <Event.WS_CLOSE: (15,)>, 'wp': <wattpilot.Wattpilot object at 0x0000xxxxxxxxC050>},(<websocket._app.WebSocketApp object at 0x0000xxxxxxxxC2F0>, None, None))
DEBUG:wattpilot.wattpilotshell:wp_handle_events(): MQTT client not yet initialized - status publishing skipped.
ERROR: Timeout while connecting to Wattpilot!

There is no hash attribute in the authRequired message :(

[mk-maddin]

Hi, you need to have a look at the hash attibute of the authRequired message. It can currently have the value pbkdf2, bcrypt or no value (which also means pbkdf2).

For a better performance, newer devices are set to bcrypt instead of pbkdf2.

@peterpoetzi : is this just some assumption for wattpilot flex or do you have some documentation / reference or done some reverse engineering on it? Thanks for any hint.

[peterpoetzi]

@mk-maddin unfortunately there is no public documentation as this is an internal api. I am part of the team that implemented it on the charger side.

[joscha82]

here is no hash attribute in the authRequired message :(

this is strange and does, unfortunately not match the hint we got (thx btw).

I could try to implement the bcrypt-based hashing based on devicetype attribute sonewhere next week, however i do rely on somebody able to test the patch with the proper hardware.

[joscha82]

here is no hash attribute in the authRequired message :(

this is strange and does, unfortunately not match the hint we got (thx btw).

I could try to implement the bcrypt-based hashing based on devicetype attribute sonewhere next week, however i do rely on somebody able to test the patch with the proper hardware.

And to add: My wattpilot on firmware 42.5 (beta) sends the hash property on auth Required Message as described by @peterpoetzi

DEBUG:src.wattpilot:Message received: {"type":"authRequired","token1":"xxxxxxxxxxxxxxxxxxxxxxx","token2":"xxxxxxxxxxxxxxxxxxx","hash":"pbkdf2"}

I

Strange that it is obviously net send by @dominiktopp device. Perhaps the official solarwatt App does not rely on the hash property but uses buildin data based on serial or devicetype to decide on the proper hashing.

[dominiktopp]

My Wattpilot is at version 42.7 (see also my debug output) with no update available according to the iPhone app.

If someone could implement hashing with pbkdf2 I could try at the weekend.
Or implement both methods (bcrypt and pbkdf2). I am no python dev but reading python and changing a method name should be possible 😄

[n0nk]

Hi, I also have Wattpilot Flex hardware (42.8 Beta Firmware) and I would be happy to help with testing or evaluating results. Attached is my current debug log.

INFO:wattpilot.wattpilotshell:Automatically connecting to Wattpilot ...
DEBUG:wattpilot:Calling event handler for event type 'Event.WP_CONNECT ...
INFO:wattpilot:Wattpilot connected
INFO:websocket:Websocket connected
DEBUG:wattpilot:Calling event handler for event type 'Event.WS_OPEN ...
DEBUG:wattpilot.wattpilotshell:wp_handle_events(event={'type': <Event.WS_OPEN: (18,)>, 'wp': <wattpilot.Wattpilot object at 0x7fbbd086b0>},(<websocket._app.WebSocketApp object at 0x7fbc623f50>,))
DEBUG:wattpilot.wattpilotshell:wp_handle_events(): MQTT client not yet initialized - status publishing skipped.
DEBUG:wattpilot:Message received: {"type":"hello","serial":"XXXXXXXX","hostname":"Wattpilot-XXXXXXXX","friendly_name":"Wattpilot","manufacturer":"fronius","devicefamily":"wattpilot","devicetype":"wattpilot_flex","devicesubtype":"wattpilot_flex_c6","version":"42.8","protocol":2,"proto":3,"secured":true}
DEBUG:wattpilot:Calling event handler for event type 'Event.WS_MESSAGE ...
INFO:wattpilot:Connected to WattPilot Serial XXXXXXXX
DEBUG:wattpilot:Calling event handler for event type 'Event.WP_HELLO ...
DEBUG:wattpilot:Message received: {"type":"authRequired","token1":"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","token2":"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"}
DEBUG:wattpilot:Calling event handler for event type 'Event.WS_MESSAGE ...
DEBUG:wattpilot:Message send: {"type": "auth", "token3": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", "hash": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"}
DEBUG:wattpilot:Calling event handler for event type 'Event.WP_AUTH ...
DEBUG:wattpilot:Message received: {"type":"authError","message":"Wrong password"}
DEBUG:wattpilot:Calling event handler for event type 'Event.WS_MESSAGE ...
ERROR:wattpilot:Authentication failed: Wrong password
DEBUG:wattpilot:Calling event handler for event type 'Event.WP_AUTH_ERROR ...
DEBUG:wattpilot:Calling event handler for event type 'Event.WS_CLOSE ...
DEBUG:wattpilot.wattpilotshell:wp_handle_events(event={'type': <Event.WS_CLOSE: (15,)>, 'wp': <wattpilot.Wattpilot object at 0x7fbbd086b0>},(<websocket._app.WebSocketApp object at 0x7fbc623f50>, None, None))
DEBUG:wattpilot.wattpilotshell:wp_handle_events(): MQTT client not yet initialized - status publishing skipped.
ERROR: Timeout while connecting to Wattpilot!

[mk-maddin]

I am trying to replicate the javascript code in python but so far no luck.
It seems the javascript does some string byte encoding differntly (Little-Endian vs. Big-Endian, other ?), but I was not able to figure it out yet...

@joscha82 maybe still helpfull (obviously password & serial are test data):

JS dependencies
npm install crypto-js
npm install bcryptjs --> https://github.com/dcodeIO/bcrypt.js

import bcrypt from "bcryptjs";
import sha256 from "crypto-js/sha256.js";

var password = 'SecurP4$$w0Rd!';
var serial = '91234567';
var iterations = 8;

//JUST FOR TEST/VERIFICATION 1:1 function 
//base64_encode from https://github.com/dcodeIO/bcrypt.js/blob/28e510389374f5736c447395443d4a6687325048/index.js#L427
var BASE64_CODE = "./ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789".split("");
function base64_encode(b, len) {
  var off = 0,
    rs = [],
    c1,
    c2;
  if (len <= 0 || len > b.length) throw Error("Illegal len: " + len);
  while (off < len) {
    c1 = b[off++] & 0xff;
    rs.push(BASE64_CODE[(c1 >> 2) & 0x3f]);
    c1 = (c1 & 0x03) << 4;
    if (off >= len) {
      rs.push(BASE64_CODE[c1 & 0x3f]);
      break;
    }
    c2 = b[off++] & 0xff;
    c1 |= (c2 >> 4) & 0x0f;
    rs.push(BASE64_CODE[c1 & 0x3f]);
    c1 = (c2 & 0x0f) << 2;
    if (off >= len) {
      rs.push(BASE64_CODE[c1 & 0x3f]);
      break;
    }
    c2 = b[off++] & 0xff;
    c1 |= (c2 >> 6) & 0x03;
    rs.push(BASE64_CODE[c1 & 0x3f]);
    rs.push(BASE64_CODE[c2 & 0x3f]);
  }
  return rs.join("");
}

password = sha256(password).toString();
console.log("password =", password);

serial = serial.padStart(16, '\0');
console.log("serial =", serial);
let serial_b642 = base64_encode(serial, 16);
console.log("serial_b64 (from function) =", serial_b642);
let serial_b64 = bcrypt.encodeBase64(serial, 16);
console.log("serial_b64 ( from bcrypt ) =", serial_b64);

let salt = [];
salt.push("$2a$");
if (iterations < 10)
salt.push("0");
salt.push(iterations.toString());
salt.push('$');
salt.push(serial_b64);
salt=salt.join('');
console.log("salt =", salt);

let hash = bcrypt.hashSync(password, salt);
hash = hash.substring(salt.length);
console.log("hash =", hash)

Python code:

import hashlib
import bcrypt
import base64
import os

# Definierte Variablen
password = 'SecurP4$$w0Rd!'
serial = '91234567'
iterations = 8

#BASE64_CODE from: https://github.com/dcodeIO/bcrypt.js/blob/28e510389374f5736c447395443d4a6687325048/index.js#L402C1-L403C80
BASE64_CODE = list("./ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789")

#encodeBase64 from https://github.com/dcodeIO/bcrypt.js/blob/28e510389374f5736c447395443d4a6687325048/index.js#L1133C17-L1133C29
#base64_encode from https://github.com/dcodeIO/bcrypt.js/blob/28e510389374f5736c447395443d4a6687325048/index.js#L427
def bcryptjs_encodeBase64(b: bytes, length: int) -> str:  
    off = 0
    rs = []
    
    if length <= 0 or length > len(b):
        raise ValueError(f"Illegal len: {length}")
        
    while off < length:
        c1 = b[off] & 0xff
        off += 1
        rs.append(BASE64_CODE[(c1 >> 2) & 0x3f])
        c1 = (c1 & 0x03) << 4
        if off >= length:
            rs.append(BASE64_CODE[c1 & 0x3f])
            break

        c2 = b[off] & 0xff
        off += 1
        c1 |= (c2 >> 4) & 0x0f
        rs.append(BASE64_CODE[c1 & 0x3f])
        c1 = (c2 & 0x0f) << 2
        if off >= length:
            rs.append(BASE64_CODE[c1 & 0x3f])
            break

        c2 = b[off] & 0xff
        off += 1
        c1 |= (c2 >> 6) & 0x03
        rs.append(BASE64_CODE[c1 & 0x3f])
        rs.append(BASE64_CODE[c2 & 0x3f])

    return "".join(rs)

password_hash_sha256 = hashlib.sha256(password.encode('utf-8')).hexdigest()
print(f"password = {password_hash_sha256}")

bserial = serial.encode("utf-8")
bserial = bserial.rjust(16, b'\x00');
print(f"bserial = {bserial}")
serial_b64 = bcryptjs_encodeBase64(bserial, 16)
print(f"serial_enc = {serial_b64}")

salt = []
salt.append("$2a$")
if (iterations < 10):
    salt.append("0")
salt.append(str(iterations))
salt.append("$")
salt.append(serial_b64)
salt=''.join(salt)
bsalt = salt.encode("utf-8")
print(f"bsalt = {bsalt}")

bpassword = password_hash_sha256.encode('utf-8')
pwhash = bcrypt.hashpw(bpassword, bsalt)

salt_length = len(salt)
pwhash_sub = pwhash[salt_length:].decode('ascii')
print(f"hash = {pwhash_sub}")

[joscha82]

@mk-maddin : yes. bcrypt does use an custom base64 alphabet and does ditch the padding with "=". Regular base64 encoding does not work with bcrypt.

This complicates things a bit. I have a PoC using passlib (only for generation of of salt) and byrypt library (for creating hash). Which creates at least a hash (hopefully the correct one).
Will take me a few days to merge the PoC with wattpilot code however.

[mk-maddin]

EDIT: seems like issue was with digit VS ascii byte conversion...

The following works fine in my tests (javascript & python deliver same hash - for digit only serial numbers; Not sure if non-digit serial numbers will be a thing in future).
As you I unfortunately do not have a Flex for testing about the hash logon but at least it delivers the same result as the javascript example of @peterpoetzi

import hashlib
import bcrypt
import base64
import os

# Definierte Variablen
password = 'SecurP4$$w0Rd!'
serial = '91234567'
iterations = 8

#BASE64_CODE from: https://github.com/dcodeIO/bcrypt.js/blob/28e510389374f5736c447395443d4a6687325048/index.js#L402C1-L403C80
BASE64_CODE = list("./ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789")

#encodeBase64 from https://github.com/dcodeIO/bcrypt.js/blob/28e510389374f5736c447395443d4a6687325048/index.js#L1133C17-L1133C29
#base64_encode from https://github.com/dcodeIO/bcrypt.js/blob/28e510389374f5736c447395443d4a6687325048/index.js#L427
def bcryptjs_base64_encode(b: bytes, length: int) -> str:  
    off = 0
    rs = []
    
    if length <= 0 or length > len(b):
        raise ValueError(f"Illegal len: {length}")
        
    while off < length:
        c1 = b[off] & 0xff
        off += 1
        rs.append(BASE64_CODE[(c1 >> 2) & 0x3f])
        c1 = (c1 & 0x03) << 4
        if off >= length:
            rs.append(BASE64_CODE[c1 & 0x3f])
            break

        c2 = b[off] & 0xff
        off += 1
        c1 |= (c2 >> 4) & 0x0f
        rs.append(BASE64_CODE[c1 & 0x3f])
        c1 = (c2 & 0x0f) << 2
        if off >= length:
            rs.append(BASE64_CODE[c1 & 0x3f])
            break

        c2 = b[off] & 0xff
        off += 1
        c1 |= (c2 >> 6) & 0x03
        rs.append(BASE64_CODE[c1 & 0x3f])
        rs.append(BASE64_CODE[c2 & 0x3f])

    return "".join(rs)

def bcryptjs_encodeBase64(s: str, length: int) -> str:
    if s.isdigit(): #numeric only serial
        vals = [ord(ch) - ord('0') for ch in s]
        b = bytes([0] * (length - len(vals)) + vals)
    else: #not sure about serials in future - fallback
        raise ValueError(f"Check serial - should be digits only")
    return bcryptjs_base64_encode(b, length)


password_hash_sha256 = hashlib.sha256(password.encode('utf-8')).hexdigest()
print(f"password = {password_hash_sha256}")

serial_b64 = bcryptjs_encodeBase64(serial, 16)
print(f"serial_b64 = {serial_b64}")

salt = []
salt.append("$2a$")
if (iterations < 10):
    salt.append("0")
salt.append(str(iterations))
salt.append("$")
salt.append(serial_b64)
salt=''.join(salt)
bsalt = salt.encode("utf-8")
print(f"bsalt = {bsalt}")

bpassword = password_hash_sha256.encode('utf-8')
pwhash = bcrypt.hashpw(bpassword, bsalt)

salt_length = len(salt)
pwhash_sub = pwhash[salt_length:].decode('ascii')
print(f"hash = {pwhash_sub}")

[dominiktopp]

@mk-maddin maybe you can push this to a new branch and change the existing wattpilot impl to use this code?
I have a Flex an can test it tomorrow or on Sunday...

[mk-maddin]

@dominiktopp @n0nk

Can you please test with: https://github.com/mk-maddin/wattpilot/tree/main
It's a absolute mess with comments and log of debugging messages; so not at any pull/merging state- but should be ok for a first test.
(Please attention with logging output - its all pw, serial, hash etc. written in plain text)

EDIT: comment & debugging messages removed; confirmed to be working by 2 independent testing people