Data Synchronisation
Mandalay’s DataSync Tool
Synchronising your data no matter what
Mandalay’s DataSync tool provides a solution for synchronising on-premise and remote PCs with a cloud-based master database and applications.
Designed as a more robust replacement for SQL Replication, DataSync even works well in low bandwidth and unstable WAN environments.
What is DataSync?
DataSync is a component of the Mandalay Facility Product Suite which links on-premise machines (generally point-of-sale or driver control stations) with the cloud-based components. It passes relevant changes from the cloud to remote sites, and changes made at those sites back to the cloud. It also initialises new sites with a complete copy of their data when initially deployed.
Designed to provide more reliable data synchronisation even in locations with low bandwidth and unstable networks. DataSync has been demonstrated to be more reliable than the traditional industry equivalent, SQL Replication, in the environments we typically see within the waste industry.
DataSync also offers substantially improved record level auditing and permanent retention of all data changes for auditing purposes.
What are the benefits
Why is DataSync better?
Easier to Maintain
Mandalay's DataSync tool is much easier to maintain.
Can be Automatically Deployed
It can be deployed and updated automatically.
Easy to Provision Replacement PCs
It is easy to provision replacement PCs to cover failures, with the option for this to be handled entirely by your IT department as needed.
Health Monitoring and Self-Healing
It is a highly available 24x7 cluster with built in health monitoring and self-healing.
Dashboard Access
You have dashboard access providing an overview of DataSync and PC health (e.g. synchronisation status, available hard drive space, etc.).
Flexibility for Future Growth
There is flexibility for future growth (e.g. customised conflict handling strategies).
Historical Records are Maintained
A historical record of all synchronised changes is maintained.
No Firewall Changes Required
DataSync functions entirely over standard HTTPS ports, so no firewall changes are required.
What makes DataSync secure
Why can you trust DataSync?
SSL Security
All DataSync communications are secured via encryption (SSL).
Extensively Tested
Extensive testing has taken place with DataSync to ensure it quickly and accurately synchronises changes.
Handles Failures Safely
DataSync is designed gracefully handle both routine (e.g. conflicts and connection outages) and unusual failure cases (e.g. network attacks and failed infrastructure).
Queue-based Approach
A queue-based approach means it is highly resilient to failures and preserves data in case of failure.
Network Operations are Authenticated
All network operations are authenticated via machine specific keys or user logins.
Data is Restricted
A machine key or login only allows access to a single client’s data.
Continually Monitored
Continual monitoring allows a proactive approach to detecting and resolving issues.
Built on Microsoft Azure
DataSync is built on the Microsoft Azure platform.
How does DataSync operate?
The DataSync client runs as a Windows service on each remote machine. Its operation can be broken down as follows:
Detect Local Changes – The local SQL Server database is monitored for changes (via SQL Change Tracking). When changes are detected they are recorded separately in a queue awaiting upload.
Upload Local Changes to the Cloud – As available, local changes are uploaded to the cloud via an Azure Service Bus Topic.
Download and Apply Changes from the Cloud – Changes are downloaded from an Azure Service Bus Subscription for the specific machine and then applied to the local database.
Process Commands – DataSync configuration and heartbeat commands may be sent to on-premise machines via the service bus.
Send Health Metrics – On-premise machines periodically report health information (CPU, HDD, memory) state to the cloud application.
It is important to note that all communication between the on-premise machine and Azure Service Bus is secured and encrypted via SSL and authenticated using a machine specific token with permissions to allow that machine to upload its changes, and download changes meant for it.
Conflicts (record modified at multiple sites at once) are resolved in the cloud, and the resolution then passed to all affected sites.
What makes it possible
How is DataSync initially deployed and future updates released?
Both initial deployment and subsequent updates are deployed via Octopus Deploy. This allows the process to be highly automated. To support this an Octopus Tentacle Windows Service will be installed to on-premise machines. This polls the Mandalay Octopus server to receive DataSync and Mandalay Facility Product Suite application updates.
Frequency and scheduling of updates will be determined by your service agreement.
Minimum requirements
What needs to be in place for DataSync to operate?
Software required on the client machines to support DataSync are as follows:
- Windows 7 SP 1 or later
- PowerShell 3.0 or later (e.g. Windows Management Framework 4.0)
- Octopus Tentacle
Additional software installed automatically during deployment may include:
- Microsoft SQL Server Express 2014 or newer
- Microsoft .net Framework 4.61
For the Octopus Tentacle Windows Service, on-premise machines will need to be able to reach the following on port 443:
- naus.com
For the DataSync Windows Service, on-premise machines will need to be able to reach the following on port 443:
- servicebus.windows.net
- naus.com
- api-gateway.naus.com
What are the limitations of DataSync?
DataSync will continue to evolve over time with new features. Currently, it has the following limitations.
- Only the Facility Product Suite Cloud Hosted Solution is targeted.
- Data is not throttled.
- DataSync does not co-exist with SQL Replication on the synchronised database.
- DataSync is not compatible with the use of triggers on the synchronised SQL database.
System impacts
What impacts will DataSync have on your systems?
DataSync should have minimal impact on CPU, memory and HDD usage.
The volume of data transferred over the network will depend on several factors. The largest impact is when loading an on-premise machine with its initial data. During normal operation, the data volume is determined by the rate of changes (new and/or updated records) created on the on-premise machine, and the volume of relevant changes made on other machines or in the cloud environment.
The following provides an example only for a small site (3412 records required for initialisation of site machine, 600 transactions performed at the site each day, with an average of 4 record changes per transaction).
Number of Records in Initialisation: 3412
Downloaded during Initialisation: 3 MB
Uploaded during Initialisation: 8 MB
Average Network Impact per Record: 12 KB
Tickets per Day: 600
Site Records Changes per Day: 2400
Site Data Uploaded per Day: 12 MB
Contact Us
Contact the team at Mandalay for more information.
Additional Resources
Where Should your Software and Data Solution be Hosted?
Future proof your waste facility with a hybrid cloud solution that consists of software that’s hosted both in the cloud and on site. With a hybrid data solution, data is generated on site and synchronised to the cloud.
How Mandalay Will Take Your Data Security & Accessibility to New Heights
Data security is an increasing focus for organisations globally, particularly for those with sensitive information that they must keep safe. But unfortunately, this is an area the waste industry has traditionally been lacking in, until now.
Leaders in data collection, information control and management.
GET IN TOUCH
