Overview
ClearScale, established in 2011, specializes in cloud system integration, application development, and managed services. As an AWS Premier Consulting Partner, they excel in Migration, DevOps, Marketing & Commerce, Big Data, and Mobile sectors. Their team, composed of certified AWS engineers, architects, and developers, carries a proven track record in delivering successful IoT projects. ClearScale designs and automates IoT platforms, develops IoT applications, and creates infrastructure that enables secure device interactions, data gathering, and insightful analytics for businesses.
Recently, ClearScale collaborated with Samsung to roll out a Cloud Solutions Platform, providing robust infrastructure and printing solutions accessible from any device, any location. To build the device management aspect of the platform, ClearScale performed a competitive analysis between AWS IoT and the existing Ejabberd messaging solution.
The primary objective was to develop a reliable printing service methodology for Samsung’s users, focusing on the critical device management component. This component facilitates authentication, messaging between devices (such as printers), and the cloud infrastructure. Additionally, it collects data from devices for analysis, helping Samsung monitor device health and usage for effective troubleshooting and proactive maintenance.
Application Overview
Defining Evaluation Criteria
In collaboration with Samsung, ClearScale established criteria to evaluate AWS IoT against Ejabberd for device management functionality. The criteria were prioritized based on Samsung’s specific business needs. While these areas are generally applicable to any IoT evaluation, the scoring methodology might differ based on unique client requirements.
The evaluation focused on two main areas: functional testing and load testing. Functional testing assessed both solutions’ core capabilities, security, and technology reach. Load testing aimed to measure availability, scalability, maintainability, performance, and reliability, with gathered metrics forming a scoring system.
Functional Testing
ClearScale initiated functional testing to ensure each system met the defined requirements before moving to load testing. A small Ejabberd environment was established alongside AWS IoT to ensure parity. Five functional tests validated that both platforms met Samsung’s needs without issues.
Load Testing
To compare the two platforms, ClearScale formulated load testing criteria which included two main scenarios: simulating peak load conditions and demonstrating system stability. The message rates were calculated based on user profiles:
- Consumer (2-3 jobs/week)
- SMB (10-20 jobs/week)
- Enterprise (150-300 jobs/week)
With a proposed distribution of 50%, 30%, and 20%, the simulations included 500,000 agents.
Calculating message averages per second involved various factors, yielding an average of 496.032 messages per second.
For maximum message rates, during busy hours (90% of jobs executed), the capacity reached about 6,000 messages per second.
Load Generation
Apache JMeter was chosen as the load generation tool due to its extensibility and community backing. Custom plugins for XMPP and MQTT protocols were developed to enable accurate testing of both solutions. These plugins facilitated secure connections and persistent connections, enhancing the realism of the load generation process.
By leveraging existing tools and extending their capabilities, ClearScale reduced the development time required for load generation.
For more insights into similar projects, check out another blog post here. Additionally, if you’re looking for authoritative content on the subject, refer to this link. For community discussions and resources, this Reddit thread is excellent.
Leave a Reply