I've got the power 🔌!

Powershop is in the process of updating all their clients to their new mobile/web application. With this change, the old import Powershop data from downloaded CSV, does not work anymore.

That needs fixing, of course.

Everything is different now

Despite moving to a modern framework and having mostly built the API already, Powershop does not have an official API as of yet.

Admittedly, they have a constant warning in their new (web)application, that lets you know that not everything is there yet.

That means that there's still some hacking around to do!

To get things working, I divided the changes up in 3 parts:

1. Be able to login
2. Be able to pull the data
3. Translate the pulled data to Home Assistant

Part 1 requires GraphQL and JWT, part 2 requires grabbing requests as the web-interface makes them, and part 3 was already largely there thanks to previous work.

As the new system uses JWT, the obvious choice was to use PyJWT. That was the easy part.

The hardest part really, was to get the required keys and such.

As the new system uses Firebase for its authentication, the API key for Firebase is needed to even be able to get started.

Turns out that everything is stored on the local machine. That makes life easier, a quick look at the local storage showed a Firebase object. With the required API Key. Step one of two completed!

Part two, the refresh token. JWT is an interesting authentication system, as in that a user logs in for a very short period, but receives 2 secrets. The first is the "access_token", that does the authentication, and the second is a "refresh_token". The refresh token gives the user the ability to request a new access_token.

A refresh token is sometimes also called a "long lived token", because it is valid for longer. It does not, however, contain all the information needed to log in, only the information needed to get a new refresh token, which can then be used to log in again.

Confusing? A little bit.

Thankfully, the before mentioned local storage object of Firebase, not only contains the API key, it also contains the refresh token and the access token! Score!

The local storage object is named something like
"firebase:authUser:aShorterGibberishStringKey:[DEFAULT]:", where the gibberish string key is the API key for Firebase.

The value of this storage object, has everything needed to get going. This is what it looks like, with sensitive values replaced:

javascript:

{
"uid": "UNIQUE_ID",
"email": "user@example.com",
"emailVerified": true,
"displayName": "John Doe",
"isAnonymous": false,
"providerData": [
{
"providerId": "password",
"uid": "user@example.com",
"displayName": "John Doe",
"email": "user@example.com",
"phoneNumber": null,
"photoURL": null
}
],
"stsTokenManager": {
"refreshToken": "aLongStringOfRandomGibberishThatIsTheRefreshToken",
"accessToken": "aThreePartString.SeparatedByDots.ItIsTheAccessToken",
"expirationTime": 12345678987654321
},
"createdAt": "1784762171932",
"lastLoginAt": "1785143998317",
"apiKey": "aShorterGibberishStringKey",
"appName": "DEFAULT"
}

The API key, the refresh token, and the access token are clearly marked.

A small bit of Python will get us a new refresh and access token. The code is slightly altered to display correctly, but the gist is the same.

python:

refresh = {
"refreshToken": payload.stsTokenManager.get('refreshToken'),
"grant_type": "refresh_token"
}
 alg = "RS256"
 result = requests.post(f"https://securetoken.googleapis.com/v1/token?key={payload.get('apiKey')}",
 data=refresh)
 decoded = json.loads(result.text)
 jwt_token = jwt.decode(decoded.get("access_token"), algorithms=["HS256", "RS256"], verify=False, options={"verify_signature": False})

This bit of code gives us a JWT token that we can use to grab data from Powershop!

I've set verify_signature to false, because I do not have the private key needed to validate that the authorization matches the signature. Luckily, that's not even needed for me to have or know. So this will do just fine.

For this part of the code, I cheated a bit...

I looked at the request done by the web portal, and pulled out the specific request and the query that it did, and simply pasted that into my application. Then, it was a matter of getting the variables, such as meter ID, ICP, property ID, et cetera, and put them in the right place. With that, I actually got a correct response back immediately!

GraphQL is one of those ways of communicating over the internet, that is excessively verbose and key/value heavy.

The response from the API has a very deeply nested response:

javascript:

{
"data": {
"account": {
"id": "",
"property": {
"edges": [
{
"node": {
"source": "Amphio",
"value": "12345.670000000000000000",
"unit": "kwh",
"readAt": "1970-01-01T12:00:00+12:00",
"metaData": {
"utilityFilters": {
"readingFrequencyType": "POINT_IN_TIME",
"readingDirection": "CONSUMPTION",
"registerId": "1",
"deviceId": "123456789",
"marketSupplyPointId": "0000123456AB1C2",
"readingQuality": "ACTUAL",
"__typename": "ElectricityFiltersOutput"
},
"statistics": [],
"__typename": "MeasurementsMetadataOutput"
},
"__typename": "MeasurementType"
},
"__typename": "MeasurementEdge"
}
]
}
}
}
}

I've left out a lot, but for every hour of a value record, there is an entire block of the "node" response above.

Reformatting everything to be a simpler structure was a lot of searching. The metadata reflects the query I made, e.g. the "readingQuality" there, is ACTUAL, because I requested only actual readings.

There really is a lot of data being sent backward and forward, for very little actual useful information. This is also why I am not a big fan of GraphQL.

This was a lot of for-looping, and figuring out, but I managed to untangle it all. In the end, it required mostly looping over all the nodes of the response, and figuring out if it was a peak, off-peak, or weekend, for the given date and time.

Aside from pulling all the data in, it was largely using existing code to get me to this stage. Including but not limited to using my old code and re-purposing it.

Once that was all done, I used a bit of my old code to push it all into Home Assistant.

And then....

It was too easy to work immediately, of course.

I made a mistake, and now my measured energy usage in Home Assistant doubled every hour... One + in the wrong place, and everything was doubling up all the time.

This wasn't hard to resolve, but it did cause me some headaches figuring out what my starting point should be, and finding the last correct values in my Home Assistant database.

All said and done, it did work! I'm currently testing it manually every day, to see if I can rely on it doing what I want it to do.

The link to the repo is above, and I'll be updating the README with instructions on how to get started with the new Powershop Lab version as soon as I have time.

As the new GQL API also supplies cost, from the day that the new Labs platform is active for the Powershop user, it also provides the cost per hour, and a standing charge.

I've decided to include the cost per hour, so it can be displayed on the Home Assistant dashboard too.It's imported the same way with the statistics importer API, and requires zero additional configuration.

Many thanks to Daniel Compton for pointing out and providing a starting point to fix some small bugs!

I've omitted the Standing Charge, as it is currently not a core feature of HA, and therefore there are multiple implementations, but none actually without hacking around in HA. I suspect a standing charge feature will be added to the dashboard sooner or later, in a less hacky way than the current options.

Therefore, I decided against it. If you want to include your standing charge, it's a static daily price you can look up on the Powershop website (It's NZ$ 1.95 for Wellington).