2024 Backend Release

2.7.1 (06-Dec-2024)

Please upgrade the Node Registration module to v23 after upgrading the backend.

caution

Important, before upgrade We have observed that some of the Cognito settings are getting reset to default after the RainMaker upgrade. Please make sure to take the backup of Cognito settings before the upgrade. Steps are documented here.

caution

Important, after upgrade

1. Please reset the default email sender to AWS SES instead of Cognito. Steps are documented here.
2. Please set the value of Cognito-assisted verification and confirmation attribute to Send SMS message if phone number is available, otherwise send email message. Steps are documented here

• New features
  • Node Registration:
    • Added an option to update information of registered nodes. A new option --update_nodes has been added to update type, model, subtype and tags for nodes that are already registered with the backend.
    • Added an option to re-register nodes. A new option --force has been added in case certificates for some registered nodes need to be re-registered (maybe due to change in certificates) along with adding information like type, model, subtype and tags.
    To use these parameters please pull the latest RainMaker Admin CLI code.
  • Admin User Management: Admins can customise the email that gets triggered to user for tags assignment verification. Refer Admin User Management APIs and Message Template APIs for more details.
  • Users: Users can attach Tags to themselves after signup. Refer User Management APIs for more details.
• Bug fixes
  • OTA: The OTA job status was "active" even when all the nodes had completed the OTA.
  • Admin User Management: Custom data in user details was getting returned as text instead of its original format. Refer Admin User Management APIs for more details.
  • Node group: Sharing of a Matter fabric was failing if the fabric has subgroups with matter nodes.
  • Tags: Dynamic Tags added during node registration were also adding the tags column with the @ prefix.
• Known issues
  • OTA Job Count: The total count of nodes in an OTA job is incorrect if the job was created with a Node Group having group query filters.
  • Node Tags and User Mapping: Tags added during node registration are removed when the user does user node mapping.
  • The OAuth only RainMaker upgrade leads to an Identity pool reset. As an effect, all logins stop working.

2.7.0 (26-Nov-2024)

Please upgrade the Node Registration module to v21 after upgrading the backend.

caution

Important, before upgrade We have observed that some of the Cognito settings are getting reset to default after the RainMaker upgrade. Please make sure to take the backup of Cognito settings before the upgrade. Steps are documented here.

caution

Important, after upgrade

1. Please reset the default email sender to AWS SES instead of Cognito. Steps are documented here.
2. Please set the value of Cognito-assisted verification and confirmation attribute to Send SMS message if phone number is available, otherwise send email message. Steps are documented here

• New features
  • User Public Profile: Added concept of public profile for users. As the profile is public, other authenticated users can have access to it. Refer User Public Profile API for more details.
  • User Account: Users will now be notified about their account deletion through email. Admin can configure DELETE_USER_CONFIRM_MESSAGE message template to be used to send this email. Refer Message Template API for more details.
  • User File Management: Users can now upload, fetch and manage custom files. Refer File Upload API for more details.
  • Admin User:
    • When querying for Users attached on a Tag, Admins can now get a Counts of mapped nodes for users with an additional query param node_counts. Refer Tag Service API for more details.
    • Admins can filter Users based on case-insensitive email prefix. Refer Admin User Management API for more details.
  • OTA Job: Users can now fetch the firmware binary for user approval job. Refer OTA Service API and User OTA APIfor more details.
  • User Node mapping: Simplified the Challenge response based user node mapping workflow. Refer User Node Association Mapping Initiate API and User Node Association Mapping verify API for more details.
  • OTA Image: Firmware binaries with size upto 200 MB are now supported through API. Refer OTA Image API for more details. Dashboard support will be available soon.
  • Added support for a direct MQTT link between node and phone app. Please get in touch with esp-rainmaker-support@espressif.com for more info.
• Bug fixes
  • Automation Trigger:
    • A user who has Primary access of node through sharing was not allowed to update and delete on automation trigger. Refer Automation Trigger and Actions API for more details.
  • User: User's refresh token wasn't getting revoked in logout. Now it can be revoked when passed in logout API. Refer User Management API for more details.
  • Device Defender: Super admin user was getting false email notification when the device comes back to normal message rate.
  • OTA: When an OTA was given to node group with more than 100 nodes, only 100 nodes were receiving the OTA.
  • Node group: Node connected status from the node details was returned as false even when the node is connected.
  • OTA Package: Package upload was failing if the file is not zip, now it will return validation error.
  • Timeseries: When a node mapped to a new User, the TS data of previous user was accessible to new user.
  • Tags: While fetching possible values for a given tag name, if the tag name itself was absent, a server error (5xx) was getting reported. Now, an empty response will be sent. Refer Tags Service API for more details.
  • Cloudwatch User: IAM User was not getting created. This user can be used for troubleshooting the cloudwatch logs. Refer Cloudwatch Log IAM user API for more details.
  • Admin User: Admin flag of internal user was not getting updated when a superadmin is downgraded to admin. Refer Admin User Management API for more details.

note

Deprecations in the upcoming releases for cost and performance optimisation:

• For all paginated admin APIs, the total field will be omitted.
• Node count summary (Active, Inactive and Not activated) will be emailed to the superadmin (Swagger).

2.6.0 (18-Oct-2024)

Please upgrade the Node Registration module to v20 after upgrading the backend.

caution

Important, before upgrade We have observed that some of the Cognito settings are getting reset to default after the RainMaker upgrade. Please make sure to take the backup of Cognito settings before the upgrade. Steps are documented here.

caution

Important, after upgrade

1. Please reset the default email sender to AWS SES instead of Cognito. Steps are documented here.
2. Please set the value of Cognito-assisted verification and confirmation attribute to Send SMS message if phone number is available, otherwise send email message. Steps are documented here

• New features
  • Admin access: Admins can now access custom data, user node mapping timestamp and fetch the various Tag Names and Values attached to various types of resources. Refer the Admin Custom data APIs, Admin Nodes API and Tags API for more details.
  • Tags:
    • Admins can now fetch list of all tag names for a given resource type and also fetch list of all possible values for given tag name. Refer the Tags APIs for more details.
    • Support for multiple languages
    • Ability to fetch the custom data of users in get users with a particular tag. Refer the Tags APIs for more details.
  • Ota Image: Now support Image name in multiple languages. Refer the OTA Image APIs for more details.
  • File Integrity Check:
    • Ota Image and File upload API: Checksum (optional) can be specified while creating the upload request. Backend computes the checksum of the uploaded files against the input value for verifying the integrity. Refer the OTA Image APIs and FileUpload APIs for more details.
  • Timeseries:
    • Data can be retrieved in descending order of time. Refer the Timeseries APIs for more details.
    • Added event trigger for timeseries data. Refer the Timeseries APIs and Event Filter APIs for more details.
  • Admin Nodes: Activation, LastStatusUpdate, LastConfigUpdate & LastActivity timestamps are added in the response. Refer the Admin Node APIs for more details.
  • Command response: API will now return success even if target node is offline. The framework delivers the command when the node comes back online. Refer the Command response APIs for more details.
  • Deployment details email and deployment settting API will now have super admin user name and dashboard url. Refer the Deployment details APIs for more details.
  • Publish Message: Expanded functionality to notify all the end user via push notifications instead of providing individual email addresses. Refer the Publish message APIs for more details.
  • RainMaker is available in the China region. Contact RainMaker Support (esp-rainmaker-support@espressif.com) to know more about this. Refer Docs to know features availability.
  • Cloud to Cloud Authentication is generally available to connect your own cloud securely to RainMaker for user management. Contact RainMaker Support (esp-rainmaker-support@espressif.com) to know more about this.
• Bug fixes
  • Device group: Sub groups were allowed under dynamic group.
  • Device defender: When deleting the device defender stack, some AWS resources were not being deleted.
  • OTA jobs: Creating jobs with duplicate names were being allowed.
  • User Node mapping: Description text of API response was incorrect.
  • MQTT statistics: Correct count was not returned when node_id and topic_name were sent in the API parameters. Refer MQTT Statistics for more details.
  • User node group: custom_data wasn't getting updated when specified with other attributes.
  • Timeseries: All of the records were discarded if the value for any one of the measures was invalid.
  • Node group: The node group update response wasn't generic for single or multiple attribute update.
  • Tags:
    • Connectivity status was not returned when node was searched by tag.
    • Secondary users were able to update the node data like tags, metadata etc.
• Other Changes
  • File upload: The file_name will be maximum of 100 characters.
  • Node group: Discard duplicate node group sharing request. Now old sharing request will be discarded if it is created for same user and same group.

Deprecations in the upcoming releases for cost and performance optimisation:

• For all paginated admin APIs, the total field will be omitted.
• Node count summary (Active, Inactive and Not activated) will be emailed to the superadmin (Swagger).

2.5.1 (10-Sept-2024)

Please upgrade the Node Registration module to v19 after upgrading the backend.

• Bug fixes
  • SMS to Indian phone numbers using TRAI registered sender Id is not working.

caution

Important, before upgrade We have observed that some of the Cognito settings are getting reset to default after the RainMaker upgrade. Please make sure to take the backup of Cognito settings before the upgrade. Steps are documented here.

caution

Important, after upgrade

1. Please reset the default email sender to AWS SES instead of Cognito. Steps are documented here.
2. Please set the value of Cognito-assisted verification and confirmation attribute to Send SMS message if phone number is available, otherwise send email message. Steps are documented here.

note

Deprecations in the upcoming releases for cost and performance optimisation:

• For all paginated admin APIs, the total field will be omitted.
• Node count summary (Active, Inactive and Not activated) will be emailed to the superadmin (Swagger).

2.5.0 (21-August-2024)

Please upgrade the Node Registration module to v19 after upgrading the backend.

• New features

  • More special characters are supported in device group. Except #$\|, all the other special characters are now allowed.
  • API Passthrough service can now be used to send data to external services using POST requests. Refer Passthrough API for more details.
  • SMS to Indian mobile numbers: Support has been added to use the sender Id registered to Indian TRAI so that the SMS can be sent to Indian phone numbers using local route. The configuration can be set via Deployment Setting API.
  • Support for a deployment setting to give read access of nodes to admins using the Deployment Setting API. This data can be fetched using the node details flag using the Get admin user nodes API.
  • Admin node data access: Admins with access to node data can now obtain timeseries data. (Swagger).
  • Support for devices to upload the files to the cloud. Refer the Command Response Communication APIs for more details.

• Bug fixes

  • Automations: Same condition could trigger an automation multiple times E. g. If weather = "Thunderstorm" has already matched and triggered an automation, if during next periodic check, it is still the same, the automation was getting triggered again. Now, it will trigger just once. It will trigger again only if weather changes to some other value and then comes back to "Thunderstorm". If the earlier re-triggering behaviour is desired, pass "retrigger":true. Refer Automation Trigger and Actions API for more details.
  • Insights package upload on public RainMaker was broken.
  • Groups with same names could not be created under different parents.
  • For the list of users returned by tags service, only user_id was available, instead of user_id and user_name.

• Other changes

  • Optimisation to process the messages sent by devices in parallel.
  • Admin custom data can be queried using user_name query parameter instead of separate user_email and user_phone_number query parameters. The user_email and user_phone_number will be deprecated soon. Refer Custom Super Data For Admins API for more details.
  • Admin user details: Admins can now get details of a User using user_id, in addition to the previously supported user_name query parameter. Refer Admin User Management API for more details.

• Known issues

  • OTA: Status counts are sometimes showing negative values.

2.4.0 (1-July-2024)

Please upgrade the Node Registration module to v18 after upgrading the backend.

• New features
  • Support for users to give Read access to their nodes to the admin using the Put API. This data can be fetched using the node details flag using the Get admin user nodes API.
  • Support for admins to fetch end users associated with a node in Get admin user nodes API.
  • Time series:
    • Support for differential data query for all aggregates in case node has reported cumulative values. This can optionally be reset at negative values. (Swagger)
    • Nodes can overwrite data by sending 'ow':true.
  • Superadmins can add tags to users. (Swagger)
  • All pending commands can be overridden by a new command response request.
  • Added Backup timestamp when Cognito app clients are backed up in Cognito backup API.
• Bug fixes
  • Numeric string keys not getting added to JSON. Impacted node params, various custom data and metadata.
  • Duplicate group details in the Group sharing push notifications event data.
  • In progress OTA status node count always being 0 in status and summary API.
  • Duplicate nodes being returned in the Get OTA status API.
  • Push notifications sent via Publish message API could fail if user had logged out from the app and logged in again.
  • Admins not able to access Get admin user nodes API.
• Other changes
  • Change in the error message when the user doesn't have any associated nodes. (Swagger)
  • Allow single quotes in the node group name. (Swagger)
  • AWS better design:
    • Point-in-time recovery for all DynamoDB tables
    • SQS messages encrypted at rest
• Matter Fabric changes
  • Support to add custom data while matter fabric creation.
• Known issues
  • Secondary users able to give node access to the admin using the Get admin user nodes API.

2.3.0 (30-May-2024)

Please upgrade the Node Registration module to v18 after upgrading the backend.

• New features

  • API pass-through service (Swagger): This offers users a seamless way to interact with external APIs using RainMaker. Additionally, it provides functionality to manage and store the necessary API configurations for calling external APIs.

  • Support to add user tags during sign-up which will then be returned in the Get user API. Admins can then filter users based on tags using the Get tags API.

  • Support to add the node to a group during user node association.

  • Added config, params, node info and tags to the Get user groups API node details. This will be useful for applications having the Home/Group view as the primary view. This API can be used instead of GET /user/nodes so that info of only relevant nodes will be fetched.

  • Get command response historical log for user API and admin API. This setting needs to be enabled using the deployment settings API.

  • Custom data for node groups with JSON rules. Can be added during group creation or updation.

  • Return user node association timestamp in the Get user nodes API.

• Bug fixes
  • Incorrect node count for an OTA job in case same node(s) are provided while OTA creation in the nodes or groups list.
  • Server error in fetching group sharing requests if node/group sharing pending requests exceed 10. (Swagger)
• Other changes
  • Allow multiple hyphens in an OTA firmware image name.
  • OTA optimisations: the OTA node status flow and marking a job finished.
  • Set plain text email format as the default delete user verification code email message template, instead of the RainMaker branding one. You should format these emails similar to sign-up emails using the Message template API.
  • Allow primary and secondary users of a node to leave a node group. Primary users can also remove the shared group from other users. (Swagger)
  • Get command response requests by command_id for the user API and admin API.
  • Made file upload metadata generic - can be string or JSON. (Swagger)
  • OAuth: Support of changing the password for Cognito OAuth users.
• Matter Fabric changes
  • Matter controller: Allow any RainMaker node to be converted to RainMaker + Matter node. Useful for Matter controllers where the Controller is not a Matter device, but requires a Matter NodeId nevertheless. (Swagger)
  • Support to get Matter NodeIds for a group in the Get user groups API separately without fetching complete node details. This will be useful for memory-constrained clients like matter controllers.
• Known issues
  • Adding custom data while creating a matter fabric does not work. The same can be added later using the Put group API.
  • Error related to Attribute mapping while signing in with third party sign-in. The issue can be solved with steps given here.

note

Starting June 01, 2024, Google plans to remove the ability to send mobile push notifications via the legacy FCM APIs. To avoid impact on your Android application, we recommend that you plan and migrate existing platform applications on or before June 01, 2024. Here are the steps to update the push notification keys: Migrate to HTTP v1 APIs steps

note

Before upgrading, if your previous version is either of [2.0.2, 2.1.0, 2.1.2], please carry out these this:

1. Check if a log group named '/aws/lambda/esp-GetUserPoolDetails' exists using steps.
2. If it doesn't exist, please create it manually.
3. If you miss this step, the ESP-RainMaker-Core stack upgrade might fail with 'Log group not found' error. You can follow step 2 then.

2.2.0 (16-May-2024)

Please upgrade the Node Registration module to v18 after upgrading the backend.

• New features
  • To support installer based (and other similar) use cases, allow primary users to:
    • Transfer the node/group to others instead of just sharing. The user will then lose access completely.
    • Change their role to secondary after sharing the node/group with others. The user will then have limited access. (Swagger)
  • OAuth-only deployment (BETA): An alternative cost-effective solution for identity management based on the standard OAuth 2.0 protocol. Currently supports only social identity providers for user sign-up. Please get in touch with the ESP RainMaker support team (esp-rainmaker-support@espressif.com) to know more about this. (Swagger)
• Bug fixes
  • Node registration fails if the existing registered node count is above 500K.
  • 'Additional Info' empty for a node's OTA started status for non force pushed jobs.

note

Before upgrading, if your previous version is either of [2.0.2, 2.1.0, 2.1.2], please carry out these this:

1. Go to AWS CloudWatch and check if a log group named '/aws/lambda/esp-GetUserPoolDetails' exists.
2. If it doesn't exist, please create it manually.
3. If you miss this step, the ESP-RainMaker-Core stack upgrade might fail with 'Log group not found' error. You can follow step 2 then.

2.1.2 (30-Apr-2024)

Please upgrade the Node Registration module to v17 after upgrading the backend.

• Bug fixes
  • Don't error if invalid user_name given in Extend session. Fixes third-party users getting logged out from the Android phone app. (Swagger). Note that user_name is anyways not required in this call. This change is only for backward compatibility.
• Other changes
  • Support for multiple weather events in weather automation triggers. However, only one event of type weather condition is allowed. (Swagger)
• Known issues
  • Upgrading from this version intermittently gives 'Log group not found' error. Creating the log group manually will fix the issue.

2.1.0 (22-Apr-2024)

Please upgrade the Node Registration module to v17 after upgrading the backend.

• New features
  • Challenge–response based authentication for user node association as an alternative to the secret-based authentication (Swagger)
  • Securely signed OTA: With this feature, admins can push securely signed OTA images to the node (Swagger). To use this feature, compatible firmware is needed. Please get in touch with the ESP RainMaker support team (esp-rainmaker-support@espressif.com) to know more about this.
• Bug fixes
  • Intermittent error while creating an admin user.
  • Issue in simple time series data if parameter name or value contains '-' character.
• Other changes
  • Push notifications for weather automation triggers (Swagger)
  • New users added by primary users now have access to the entire simple time series data irrespective of when the node was shared with them.
  • Minor performance and cost optimisations
• Known issues
  • Automation action(s) triggered every time the specified event condition is met, regardless of the node's current state. Especially for weather automations, wherein the weather status is fetched periodically, this can cause repeated triggers.
  • Extend session API gives an error if username is not a valid email or phone number, even though not used internally. This logs out third-party users from the Android phone app.
  • Upgrading from this version intermittently gives 'Log group not found' error. Creating the log group manually will fix the issue.

note

Deprecations in the upcoming releases for cost and performance optimisation:

• For all paginated admin APIs, the total field will be omitted.
• Node count summary (Active, Inactive and Not activated) will be emailed to the superadmin (Swagger).

2.0.2 (28-Feb-2024)

Please upgrade the Node Registration module to v17 after upgrading the backend.

• Bug fixes
  • Intermittent issue of superadmin not getting created in a fresh deployment
  • Node not getting pending command requests (in command-response workflow) when explicitly queried for
  • Subgroup deletion leaving a dangling pointer to it causing failure while querying its parent group(s)
  • CORS issue in the Upload files API
• Other changes
  • Allow specifying a description while creating an OTA job, which end users can also see for user approved OTA workflows. This will be useful for better communication of firmware release notes. (Swagger)
  • Expanded weather based automation triggers for multiple weather conditions. Find the full list in the Swagger
  • Special characters now allowed in tags (Swagger). Pull the latest RainMaker Admin CLI before proceeding.
  • Improved accuracy of Statistics by periodic reconciliation
• Matter Fabric changes
  • Fixed user deletion failing when the user has matter nodes
  • Matter-only deployments: If custom user id is not specified in the header, use the authorisation header. Fixes issues of dashboard access
• Known issues
  • Upgrading from this version intermittently gives 'Log group not found' error. Creating the log group manually will fix the issue.

note

Deprecations in the upcoming releases for cost and performance optimisation:**

• For all paginated admin APIs, the total field will be omitted
• Node count summary (Active, Inactive and Not activated) will be emailed to the superadmin (Swagger)

2.0.1 (6-Jan-2024)

Please upgrade the Node Registration module to v16 after upgrading the backend.

• New features
  • Simple time series data (Swagger): This uses AWS DynamoDB rather than AWS Timestream.
    • This helps to reduce the cost in cases where the raw values are adequate and no aggregates, such as min/max/average, are required
    • Needs the relevant support to be enabled in the firmware as well
  • Weather based automation triggers: 'clear_sky' is now enabled as a weather condition (Swagger)
• Bug fixes
  • Duplicate automation triggers getting added in paginated response
  • Immediate query on a deleted node group giving inconsistent response since the deletion is asynchronous
  • Daylight automations for the first day after creation getting missed
• Other changes
  • Minimum length for Device group and automation trigger name is now one character
  • Optimised weather and daylight automation triggers processing
• Matter Fabric changes
  • Matter multi controller support (Swagger)