PointClickCare - Case Study
Ensured the safety and security of the patient data processed by the Analytics platform.
PointClickCare is the leading cloud-based software platform for the senior care market. Its goal is to help healthcare providers meet the challenges of senior care by enabling them to achieve the business results that matter – enriching the lives of their residents, improving financial and operational health, and mitigating risk. Since its introduction into the long-term post-acute care (LTPAC) market in 2000, the company now has over 10,000 senior care organizations using its software every day, enabling a more coordinated and collaborative approach to care across the senior care continuum.
PointClickCare has developed an Analytics platform that aggregates data from various data sources and provides its clients with insights by means of reporting and analytics tools. The platform is built using technologies such as Cloudera Hadoop for processing unstructured and large data sets, ETL for loading and transforming data coming from various SQL sources, PostgreSQL for data warehousing and data marts, RabbitMQ for data ingestion, and Tableau for BI & Analytics. The platform has been deployed in production for a pilot program for US customers in AWS.
Goal
- Architect and implement secure private network
- Architect and deploy wide range of analytics tools
Solution
- TriNimbus DevOps on AWS
- Automated provisioning
Outcome
- Fully meet HIPAA requirements
- Implement automation and a robust continuous delivery pipeline
We needed an AWS partner who had the software engineering knowledge along with the ability to architect and deploy a wide range of analytics tools.
Challenge
Due to the nature of data ingested into the platform – patient data collected from medical institutions around North America – , US deployment in AWS needed to comply with the HIPAA certification requirements. To achieve this, particular constraints on how AWS technologies are used had to be followed and PointClickCare decided to work with TriNimbus Technologies as an implementation partner to ensure this need was handled properly. TriNimbus is an AWS Consulting Partner with experience in implementing environments that have to meet PCI, HIPAA and other compliance requirements.
PointClickCare decided to adopt a Development Operations (DevOps) culture internally and a significant effort has been put into implementing deployment automation tools using Chef, an open source software tool. The expectation was that the same tool chain would be leveraged for the production deployment in AWS, while still conforming to policies like separation of duties and other HIPAA requirements. PointClickCare requested that TriNimbus work with its internal teams in productizing the deployment automation and building a continuous delivery pipeline for AWS, without impacting the ability to use the automation outside production or even outside AWS.
AWS and Partner Solution
To address the specific constraints required for HIPAA compliance, AWS provides a range of capabilities, including: dedicated hosts, secure networking, VPN and VPC peering, secure and encrypted S3 storage, server side EBS volume encryption, SSL load balancing, HSM-backed key management service, private managed DNS, etc. TriNimbus was able to design and implement a secure private network for hosting the infrastructure running the Analytics platform. As well, TriNimbus was able to isolate all required services inside the network, while providing a secure connection with the main PointClickCare production data centre, to enable the data loading and ingestion processes as well as SSO with the rest of the PointClickCare web application services.
“We needed an AWS partner who had the software engineering knowledge along with the ability to architect and deploy a wide range of analytics tools,” said Hiep Vuong, VP of Technology Delivery at PointClickCare. “They continue to be an essential part of this project’s go-to-market.”
To support the DevOps processes, TriNimbus worked with the PointClickCare development and build & integration teams to productize the deployment tool chain, as well as implement a continuous delivery pipeline which supports the separation of duties policies, while still allowing collaboration between the pipeline and application developers and the operations engineers. The pipeline implements a discovery mechanism for configuring the overall stack, appropriate secrets management, automated DNS management, etc. It also enables integration with the PointClickCare release process, allowing the development team to iterate fast and push more frequent releases on their end, while allowing the operations team to take responsibility for the final validation and update of the production environment.
Some of the challenges that the TriNimbus and PointClickCare teams had to overcome in the production process
of the delivery pipeline include:
- Implementing a multi- availability zone (AZ) fault-tolerant and self-healing network and network services like NAT gateways, host-to-site and site-to-site VPN, etc.
- Implementing a one-way only connectivity from the production network to shared resources outside that network for deploying cookbooks and packages.
- Implementing a private and fully managed DNS (with reverse host lookup support) using Route 53, while sharing it across VPN for use by the integration services.
- Managing credentials and other secrets required by the Chef cookbooks to configure the software without exposing those secrets outside the production environment.
- Separating environment-specific and application configuration and ensuring that the discovery mechanism can handle the former properly between environments.
- Providing secure and reliable delivery of keys, SSL key stores, etc. to all of the nodes to ensure appropriate in-transit encryption is used across all services.
On top of the continuous delivery pipeline, TriNimbus and PointClickCare were able to work together to implement automated provisioning and bootstrapping using a combination of Cloud Formation and Chef-based provisioning. This resulted in a fully automated infrastructure that is managed as code and can be used to build additional sandbox, staging or pre-production environments, or to rebuild the main production environment in the event of a disaster or in the event that moving to another AWS region is needed.
Results & Benefits
By leveraging AWS, PointClickCare was able to meet the HIPAA requirements fully and to ensure the safety and security of the patient data processed by the Analytics platform. As use of the platform becomes more frequent and more data is brought into it, the organization also is able to scale the platform to handle the high processing and low latency data replication demand of the Hadoop-based ingestion service. This is done by using an appropriate combination of: compute and I/O intensive instances running on dedicated hardware put close together in placement groups; the high query load on the PostgreSQL data warehouse, by using provisioned IOPS and enhanced networking; and bandwidth needs for the data modeling and analytics operations performed by Tableau, by optimizing the instance types for running the Tableau and Analytics application servers.
In addition to the benefits on the infrastructure side, PointClickCare was able to take advantage of various managed services and tools to reduce the operations cost and improve the automation of the deployment. These include VPC, S3, IAM, Route 53, ELB, AutoScaling and CloudFormation. By combining the AWS tools with Chef, PointClickCare was able to accomplish a high degree of automation and implement a robust continuous delivery pipeline used both by development and operations in a true DevOps fashion.
Working with TriNimbus has proven to be very valuable for PointClickCare, not only because TriNimbus was able to bring experienced Solution Architects to design and implement a secure and fault-tolerant base infrastructure, but PointClickCare was able to tap into the vast DevOps experience within its organization and join forces together in productizing the automation tool chain, so it could be extended into production along with separation of duty policies. The working relationship has been valuable to TriNimbus and AWS as well. Working with PointClickCare has been a great opportunity, as it is an agile company, which has embraced DevOps principles and is clear in its vision to extend those principles to work in a SaaS platform model under heavy compliance and IT best practice constraints.
