As the world progresses towards digitalization, ensuring cloud infrastructure security and robust management becomes imperative, especially for data-centric organizations like BigID. With an expansive digital footprint encompassing many cloud assets, accomplishing seamless and error-free cloud security management is paramount for BigID. Embracing the philosophy of Infrastructure as Code (IaC) and harnessing the capabilities of GitLab, BigID is spearheading an era of automated cloud security. This blog delves into the depths of this automation journey— explaining the rationale behind it, its benefits, and the detailed process involved.

Confronting Security Challenges

At the epicenter of BigID’s operations lies the mammoth task of managing vast quantities of data. The larger the scale of data handled, the higher the risk and probability of encountering potential security threats. In such a landscape, the importance of real-time threat detection and response escalates.

Historically, security measures were manually updated and deployed, a time- and resource-intensive process that was subject to human error. With the evolution of technology, the realization dawned that automating these processes using IaC principles could provide a more efficient, reliable, and error-free way to maintain an unwavering and consistent security posture.

Enhance Cloud Data Security Today

Embracing Infrastructure as Code (IaC) with GitLab

GitLab has become a software development standard across multiple industries, offering a comprehensive and integrated platform with features like source code management and CICD tools. It’s these robust capabilities that BigID has decided to leverage to automate its cloud security posture.

Shifting away from a traditional CICD pipeline with various stages, BigID adopted the IaC approach for configuring and securing its cloud infrastructure. IaC fundamentally transforms how computer data centers are managed and provisioned. Instead of resorting to physical hardware configuration or interactive configuration tools, the IaC philosophy endorses using machine-readable definition files.

How does this play out in the GitLab environment? BigID stores these definition files in a GitLab repository. When policies defining the desired state of the cloud infrastructure are updated and committed to the repository, GitLab’s CICD tools come into play. They automatically use these files to create, update, or delete the cloud infrastructure as necessary. This modus operandi enhances efficiency while minimizing the risk of error, a perfect blend for complex operations.

Configuring Cloud Infrastructure

To harness the full potential of GitLab’s capabilities for automating its cloud security posture, BigID’s process begins with the creation and commitment of policy files to a repository. These policy files sketch out the desired state of the cloud infrastructure. Once these policies are committed to the repository, GitLab’s CICD tools spring into action and automatically apply these changes to the cloud infrastructure.

In our architecture, a repository houses all CloudCustodian policies. Whenever an approved merge request occurs, it triggers a pipeline, creating Runners for carrying out specific jobs. These jobs outline the types of containers to be built and the tasks they will perform on a per-account and region basis. Depending on the job, the constructed containers, armed with the updated CloudCustodian policies, are pushed to the ECR registry. AWS ECS then initiates a task to pick up these policies that execute and enact remediation actions, utilizing AWS Fargate Spot Instances for execution. This process adheres to a hub and spoke model, where the container running CloudCustodian employs an AWS role linked to the container to assume specific permissions in the target accounts and regions to execute the policy. Upon completion, it forwards the execution logs to respective CloudWatch Log Groups.

One of the remarkable advantages of this approach lies in its adaptability and flexibility. BigID can tailor policies that cater to its unique operational requirements and evolving threat landscape. For instance, the policies could mandate that all data be encrypted at rest. They might also restrict access to sensitive resources, bolstering the security framework.

Within the context of cloud security management, there are two types of policies, OneTimePolicies and ContinuousPolicies, which play a critical role. OneTimePolicies are designed for single, ad-hoc policy changes. These are especially useful when a specific change is needed, such as a temporary modification in response to a unique threat or change in the cloud environment. On the other hand, ContinuousPolicies are implemented for policies that run on a regularly scheduled interval. These policies are crucial for maintaining an ongoing, automated, consistent security posture. This enables BigID to conduct regular checks and updates automatically, ensuring that the cloud infrastructure adheres to the desired state defined by the policies. In this way, OneTimePolicies and ContinuousPolicies work together, allowing for immediate, ad-hoc adjustments and consistent, scheduled checks, providing a comprehensive and flexible cloud security solution.

By leveraging the principle of Infrastructure as Code and harnessing the capabilities of GitLab’s CICD tools, BigID has achieved a significant feat in cloud security management. We have effectively automated our cloud security posture, ensuring constant updates that provide fortified protection against potential threats while slashing the risk of human error to a bare minimum. Be sure to check out our publication Accelerating Threat Detection Through Engineering for additional CICD use cases!

See BigID in Action

Lessons Learned

One of the major roadblocks we encountered during this project was optimally utilizing Amazon AWS infrastructure through our GitLab CICD pipeline. The vast array of services offered by AWS presented a steep learning curve. It was challenging to integrate and manage these services within the pipeline efficiently. Additionally, ensuring the consistent, automated deployment of our security policies across various AWS accounts proved complex, leading to a slowdown in development. Despite these challenges, the team overcame these issues by meticulously understanding each AWS service, how the CICD pipeline worked with each service, and continuously testing and iterating our configurations. This experience emphasized the importance of in-depth knowledge and proper planning when dealing with complex cloud platforms.

Closing Thoughts

Beyond this, the approach liberates valuable resources within BigID, allowing us to channel our efforts toward more strategic tasks and customer-centric initiatives. Automation not only enhances security but also promotes a culture of proactivity within the organization. This proactivity manifests itself in the ability to detect and mitigate potential vulnerabilities well before they have the chance to cause any damage. It’s akin to having an automated security guardian that never sleeps, keeping a vigilant watch over the cloud infrastructure around the clock.

Furthermore, this innovative approach to cloud security is intrinsically scalable. As BigID continues to expand its operations and manage increasingly large volumes of data, the IaC principles can easily be adapted to cater to this growth. This scalability ensures that regardless of how much BigID grows, its cloud security posture remains robust and reliable, ready to face any challenge that comes our way.

But the journey doesn’t end here. The digital landscape is perpetually evolving, with new security threats emerging. However, with an automated cloud security posture, BigID is well-equipped to stay ahead of the curve. We can adapt our security measures in real-time to tackle these threats, ensuring we remain a step ahead at all times.

The journey of automating cloud security at BigID is a testament to the innovative ways organizations can embrace technology to enhance their security posture. By coupling the principles of Infrastructure as Code with GitLab’s CICD tools, BigID is not just improving its defenses but also paving the way for a future where automation plays a central role in cloud security. In this constantly changing digital world, automation is no longer just a nice-to-have; it’s an absolute necessity, a cornerstone for organizations striving to provide secure, reliable, and efficient services.

To learn more about BigID’s on-going cloud security efforts and automated data discovery—get a free 1:1 demo today.