Policy Owner: Principal Engineer
Effective Date: 2023-05-01
Quick Reference
If you believe a security incident has occurred or is in progress, contact the CloudCard support, the Principal Engineer, or the Managing Director.
Key contacts | CloudCard Support
Luke Rettstatt / Managing Director
Anthony Erskine / Principal Engineer
|
Key Locations | Primary Office: 1103 Wise Street, Lynchburg, VA 24504 Online Meeting Room: |
Purpose
This document establishes the plan for managing information security incidents and events, and offers guidance for employees or incident responders who believe they have discovered, or are responding to, a security incident.
Scope
This policy covers all information security or data privacy events or incidents.
Incident and Event Definitions
A security event is an observable occurrence relevant to the confidentiality, availability, integrity, or privacy of company controlled data, systems or networks.
A security incident is a security event which results in loss or damage to the confidentiality, availability, integrity, or privacy of company controlled data, systems or networks.
Incident Reporting & Documentation
Reporting
If a CloudCard employee, contractor, user, or customer becomes aware of an information security event or incident, possible incident, imminent incident, unauthorized access, policy violation, security weakness, or suspicious activity, then they shall immediately report the information using one of the communication channels listed in the Quick Reference section of this document.
Reporters should act as a good witness and behave as if they are reporting a crime. Reports should include specific details about what has been observed or discovered.
Severity
CloudCard Customer Support team shall monitor incident and event tickets and shall assign a ticket severity based on the following categories.
S3/S4 - Low and Medium Severity
Issues meeting this severity are simply suspicions or odd behaviors. They are not verified and require further investigation. There is no clear indicator that systems have tangible risk and do not require emergency response. This includes lost/stolen laptop with disk encryption, suspicious emails, outages, strange activity on a laptop, etc.
S2 - High Severity
High severity issues relate to problems where an adversary or active exploitation hasn’t been proven yet, and may not have happened, but is likely to happen. This may include lost/stolen laptop without encryption, vulnerabilities with direct risk of exploitation, threats with risk or adversarial persistence on our systems (e.g.: backdoors, malware), malicious access of business data (e.g.: passwords, vulnerability data, payments information).
S1 - Critical Severity
Critical issues relate to actively exploited risks and involve a malicious actor or threats that put any individual at risk of physical harm. Identification of active exploitation is required to meet this severity category.
Escalation and Internal Reporting
The incident escalation contacts can be found in the Quick Reference section above
S1 - Critical Severity: S1 issues require immediate notification to the Principal Engineer and the Managing Director.
S2 - High Severity: A customer support ticket must be created via the support inbox and the Principal Engineer must also be notified and provided with a link to the ticket.
S3/S4 - Medium and Low Severity: A customer support ticket must be created via the support inbox and assigned to the Engineering Team for response.
Documentation
All reported security events, incidents, and response activities shall be documented and adequately protected in HelpScout.
A root cause analysis may be performed on all verified Critical security incidents. A root cause analysis report shall be documented and referenced in the incident ticket. The root cause analysis shall be reviewed by the Principal Engineer who shall determine if a post-mortem meeting will be called.
Incident Response Process
For critical issues, the response team will follow an iterative response process designed to investigate, contain exploitation, eradicate the threat, recover system and services, remediate vulnerabilities, and document a post-mortem report including the lessons learned from the incident.
Summary
Event reported
Triage and analysis
Investigation
Containment & neutralization (short term/triage)
Recovery & vulnerability remediation
Hardening & Detection improvements (lessons learned, long term response)
Detailed
Principal Engineer will manage the incident response effort
If necessary, a central “War Room” will be designated, which will be a physical room for any collocated employees, and the Online Meeting Room listed in the Quick Reference section of this document
A recurring Incident Response Meeting will occur at regular intervals until the incident is resolved
Legal and executive staff will be informed as required
Incident Response Meeting Agenda
Update Incident Ticket and timelines
Document new Indicators of Compromise (IOCs)
Perform investigative Q&A
Apply emergency mitigations
External Reporting / Breach Reporting
Plan long term mitigations
Document Root Cause Analysis (RCA)
Additional items as needed
Special Considerations
Internal Issues
Issues where the malicious actor is an internal employee, contractor, vendor, or partner requires sensitive handling. The incident manager shall contact the Managing Director directly and will not discuss with other employees. These are critical issues where follow-up must occur.
Compromised Communications
Incident responders must have text message alerts arranged before listing themselves as incident members. If there are IT communication risks, an out of band solution will be chosen, and communicated to incident responders via cell phone call.
Root Account Compromise
If an AWS root account compromise is known or expected, refer to the playbook in Appendix: PlayBook - Root Account Compromise.
Additional Requirements
Suspected and reported events and incidents shall be documented
Suspected incidents shall be assessed and classified as either an event or an incident
Incident response shall be performed according to this plan and any associated procedures.
All incidents shall be formally documented, and a documented root cause analysis shall be performed
Incident responders shall collect, store, and preserve incident-related evidence in accordance with industry guidance and best practices such as https://csrc.nist.gov/publications/detail/sp/800-86/final
Suspected and confirmed unauthorized access events shall be reviewed by the Incident Response Team. Breach determinations shall only be made by the Principal Engineer.
CloudCard shall promptly and properly notify customers, partners, users, affected parties, and regulatory agencies of relevant incidents or breaches in accordance with CloudCard policies, contractual commitments, and regulatory requirements, as determined by the Managing Director in consultation with legal counsel.
This Incident Response Plan shall be reviewed and formally tested at least annually, as described in Appendix: Preparedness Testing. Results of IR plan testing activities including findings and lessons learned will be formally documented and maintained to support security, compliance and audit requirements
External Communications and Breach Reporting
The Managing Director shall confer with the Principal Engineer and legal counsel teams in the event of unauthorized access to company or customer systems, networks, and/or data. The Managing Director shall then, in consultation with legal counsel, determine if breach reporting or external communications are required. Breaches shall be reported to customers, consumers, data subjects and regulators without undue delay and in accordance with all contractual commitments and applicable legislation. The Customer Support team will handle the communications at the direction of the Managing Director.
No personnel may disclose information regarding incident or potential breaches to any third party or unauthorized person without the approval of the Managing Director.
All statements about the breach will be honest, and no key details will be withheld that would help consumers protect themselves and their information. No data will be publicly shared if it might put consumers at further risk.
The customer support team will anticipate questions that people will ask and articulate answers to those questions. The website will be updated with answers to questions in clear plain english so as to communicate in a concise and effective manner. As call and email volume increases, the customer support team will prioritize these questions on the website and add any additional entries if necessary.
Mitigation and Remediation
The Managing Director, in consultation with the Principal Engineer and legal counsel, shall determine any immediate or long term mitigations or remedial actions that need to be taken as a result of an incident or breach. In the event that mitigations or remedial actions are needed, the Managing Director shall direct personnel with respect to planning, communicating and executing those activities.
Cooperation with Customers, Data Controller and Authorities
As needed and determined by Managing Director in consultation with legal counsel, the company shall cooperate with customers, Data Controllers and regulators to fulfill all of its obligations in the event of an incident or data breach.
After Action Review / Lessons Learned
CloudCard shall conduct a lessons learned session after the resolution of any security incident, during which CloudCard employees and key stakeholders shall take stock of the incident; get to the root of how and why it happened; evaluate how well the IRP worked to resolve the issue; and identify improvements that need to be made.
The After Action Review will consist of five steps:
Identify and collect all comments and recommendations that may be useful for future projects.
Document all findings and share them with key stakeholders.
Analyze and organize all documentation for future application.
Store documentation in a repository that can be accessed by all key stakeholder.
Retrieve documentation for use on current or future incidents.
Several key questions will be asked and answered:
What security, system, and/or process weakness were undercovered as a result of this incident whether or not they contributed to the breech?
How can we improve the IRP?
What parts of the icident response process worked well? Did we do anything positive that needed to be added to the IRP?
Roles & Responsibilities
Every employee and user of any CloudCard information resources has responsibilities toward the protection of the information assets. The table below establishes the specific responsibilities of the incident responder roles.
Response Team Members
Role | Responsibility |
Incident Manager | The Incident Manager is the primary and ultimate decision maker during the response period. The Incident Manager is ultimately responsible for resolving the incident and formally closing incident response actions. The Incident Manager will be the Principal Engineer, unless otherwise directed by the Managing Director. See the Quick Reference section for appropriate contact information. Responsibilities: Ensuring the right people from all functions are actively involved as appropriate. Communicating status updates to the appropriate person or teams at regular intervals. Resolving incidents in the immediate term. Determining necessary follow-up actions. Assigning follow-up activities to the appropriate people. Promptly reporting incident details which may trigger breach reporting, in writing to the Managing Director. |
Incident Response Team (IRT) | The individuals who have been engaged and are actively working on the incident. All members of the IRT will remain engaged in incident response until the incident is formally resolved, or they are formally dismissed by the Incident Manager. |
Customer Support | Responsible for executing communications to internal and external parties as directed by the Managing Director. Anticipate questions that people will ask and maintain central (and if appropriate, public) documentation of these answers. |
Engineers | Qualified engineers will be placed into the on-call rotation and may act as the Incident Manager (if primary resources are not available) or a member of the IRT when engaged to respond to an incident. Engineers are responsible for understanding the technologies and components of the information systems, the security controls in place including logging, monitoring, and alerting tools, appropriate communications channels, incident response protocols, escalation procedures, and documentation requirements. When Engineers are engaged in incident response, they become members of the IRT. |
Users (Employees and contractors of CloudCard) | Users are responsible for following policies, reporting problems, suspected problems, weaknesses, suspicious activity, and security incidents and events. |
Customers | Customers are responsible for reporting problems with their use of CloudCard services. Customers are responsible for verifying that reported problems are resolved. |
Legal Counsel | Responsible, in conjunction with the Managing Director, for determining if an incident presents legal or regulatory exposure as well as whether an incident shall be considered a reportable breach. Counsel shall review and approve in writing all external breach notices before they are sent to any external party. |
Managing Director | Responsible, in consultation with Legal Counsel, for determining if an incident shall be considered a reportable breach, and shall review and approve in writing all external breach notices before they are sent to any external party. CloudCard shall seek stakeholder consensus when determining whether a breach has occurred. The CloudCard Managing Director shall make a final breach determination in the event that consensus cannot be reached. |
Management Commitment
CloudCard management has approved this policy and commits to providing the resources, tools and training needed to reasonably respond to identified security events and incidents with the potential to adversely affect the company or its customers.
Exceptions
Requests for an exception to this Policy must be submitted to and Principal Engineer for approval. Exceptions shall be documented.
Violations & Enforcement
Any known violations of this policy should be reported to the Managing Director. Violations of this policy may result in immediate withdrawal or suspension of system and network privileges and/or disciplinary action in accordance with company procedures up to and including termination of employment.
Version | Date | Description | Author | Approved by |
1.0 | 2023-03-28 | First Version | Ryan Heathcote | Tony Erskine |
Appendix: Incident Collection Form
General Information | ||||
Incident Detector’s Information | ||||
Name: | Date and Time Detected: | |||
Title: | ||||
Phone: | Location Incident Detected From: | |||
E-mail: | ||||
Additional Information: | ||||
Incident Summary | ||||
Type of Incident Detected: | ||||
Denial of Service | Unauthorized Use | Espionage | Probe | Hoax |
Malicious Code | Unauthorized Access | Other: | ||
Incident Location: | ` | |||
Site: | ||||
Site Point of Contact: | ||||
Phone: | ||||
Email: | ||||
How was the Incident Detected: | ||||
Additional Information: | ||||
Location(s) of affected systems: | ||||
Date and time incident handlers arrived at site: | ||||
Describe affected information system(s) (one form per system is recommended): | ||||
Hardware Manufacturer: | ||||
Serial Number: | ||||
Is the affected system connected to a network? | Yes | No | ||
Describe the physical security of the location of affected information systems (locks, security alarms, building access, etc.): | ||||
Isolate affected systems: | ||||
Approval to removal from network? | Yes | No | ||
If YES, Name of Approver: | ||||
Date and Time Removed: | ||||
If NO, state the reason: | ||||
Backup of Affected System(s): | ||||
Last System backup successful? | Yes | No | ||
Name of persons who did backup: | ||||
Date and time last backups started: | ||||
Date and time last backups completed: | ||||
Backup Storage Location: | ||||
Incident Eradication: | ||||
Name of persons performing forensics: | ||||
Was the vulnerability (root cause) identified: | Yes | No | ||
Describe: | ||||
How was eradication validated: | ||||
Appendix: Playbook – AWS Root Account Compromise
Incident Response Runbook – Root Usage
Objective
The objective of this runbook is to provide specific guidance on how to manage Root AWS account usage. This runbook is not a substitute for an in-depth Incident Response strategy. This runbook focuses on the IR lifecycle:
Establish control.
Determine impact.
Recover as needed.
Investigate the root cause.
Improve.
The Indicators of Compromise (IOC), initial steps (stop the bleeding), and the detailed CLI commands needed to execute those steps are listed below.
Assumptions
CLI configured and installed.
Reporting process is already in place.
Trusted Advisor is active.
Security Hub is active.
Indicators of Compromise
Activity that is abnormal for the account.
Creation of IAM users.
CloudTrail turned off.
Cloudwatch turned off.
SNS paused.
Step Functions paused.
Launching of new or unexpected AMIs.
Changes to the contacts on the account.
Steps to Remediate – Establish Control
AWS documentation for a possible compromised account calls out the specific tasks listed below. The documentation for a possible compromised account can be found at: What do I do if I notice unauthorized activity in my AWS account?
Contact AWS Support and TAM as soon as possible.
Change and rotate Root password and add an MFA device associated with Root.
Rotate passwords, access/secret keys, and CLI commands relevant to remediation steps.
Review actions taken by the root user.
Open the runbooks for those actions.
Close incident.
Review the incident and understand what happened.
Fix the underlying issues, implement improvements, and update the runbook as needed.
Further Action Items – Determine Impact
Review created items and mutating calls. There are may be items that have been created to allow access in the future. Some things to look at:
IAM Users.
S3 buckets.
Elastic Beanstalk Environments
RDS Clusters
Secrets Manager
QuickSight Dashboards
Route53
Appendix: - Preparedness Testing
At least once every 12 months, CloudCard shall test the IRP with a tabletop exercise to validate the IRP and organizational and system preparedness. Incident response plan testing should address:
Communication – Testing of the processes that facilitate communication between incident handlers will ensure:
Up-to-date contact information between internal and external stakeholders (e.g., law enforcement, third-party security providers)
Working systems for incident escalations
Industry-standard encryption of the communication devices
Available facility for communication and incident management
Analysis tools – Similarly, incident response testing should ensure that the tools used to conduct threat analysis are functioning optimally. The hardware and software used in the incident analysis phase include:
Forensic workstations and backup devices
Laptops for data analysis and report generation
Printers for all documentation
Removable storage media
Analysis resources – There should also be resources available for all incident analysis processes. Incident response plan testing will ensure up-to-date:
Documentation for all security protocols
Network diagrams and critical asset inventory