Study Guides/AWS Cybersecurity Notes/Parsing logs and events - Revision history

Mflavell: /* Storing and searching in Amazon Opensearch */

2026-06-13T00:14:56Z

Storing and searching in Amazon Opensearch

← Older revision		Revision as of 00:14, 13 June 2026
Line 248:		Line 248:

	::* Application monitoring		::* Application monitoring


			Why use opensearch:


			::* Fully managed

			::* Ability to access all data

			::* Secure - IAM and SAML can secure it

			::* Scaleable

			::* Seamless intergration

2026-06-12T00:46:11Z

Storing and searching in Amazon Opensearch

← Older revision		Revision as of 00:46, 12 June 2026
Line 226:		Line 226:

	= Storing and searching in Amazon Opensearch =		= Storing and searching in Amazon Opensearch =

			Opensearch is commonly used on log protection and auditing.

			:: Opensearch is probsably an exam answer!!!


			Opensearch is:

			:: A search an analyitics engine complatable wirth Elasticsearch

			:: Can create dashboards - Opensearch dashboards.

			:: Handles large amounts of data.


			It will give you:

			::* Log analytics

			::* Full text searching

			::* Application monitoring

2026-06-12T00:29:23Z

← Older revision		Revision as of 00:29, 12 June 2026
Line 150:		Line 150:

	A subscription filter can match a pattern to keywords.		A subscription filter can match a pattern to keywords.


			= Using Amazon Kinesis with Logs =


			This is a fully managed service that can scale automatically.

			:: The system scales up and down automatically.

			:: Data can be encrypted.


			Kinesis facts to remember:

			:* It is serverless.

			:* Can automatically scale.

			:* It can transform data without other services.

			:* Can deliver data to:

			::* Opensearch

			::* Redshift

			::* Splunk

			::* S3


			The data transofrmation feature can be used to change the format of data, for example CSV to JSON.

			:: Transformation is useful for sending to Opensearch.


			Kenisis data firehose can be used to move logs between CloudWatch and S3.


			= Amazon Athena =

			Athena allows ad-hoc queries on data sets.

			:: It is serverless and can analize data stored in S3 - A big plus for logging.

			:: Queries can be written in SQL


			Athena:

			::* Uses S3 storage - no need for compute storage (used for SQL database).

			::* Serverles

			::* Pay per scan - cost savings.

			::* Supports a few log formats:

			::: Apache Web Logs

			::: CSV / TSV files

			::: JSON Files

			::: Parquet

			::: ORC



			== Transforming data ==

			:: Amazon Glue Data Catalog can transform data for Athena


			= Storing and searching in Amazon Opensearch =

2026-06-10T00:10:41Z

Cloudwatch log subscription filters

← Older revision		Revision as of 00:10, 10 June 2026
Line 133:		Line 133:

	= Cloudwatch log subscription filters =		= Cloudwatch log subscription filters =

			These allow you to stream log events from a cloudwatch log group.

			Can stream to:

			::Lambda Function

			::Data Firehose

			::: Fire

			::SNS Topic


			Can be used only to send logs that contain a spefific message - example error.

			A subscription filter can match a pattern to keywords.

2026-06-09T00:32:17Z

← Older revision		Revision as of 00:32, 9 June 2026
Line 117:		Line 117:

	Idealy, use a bucket just for logs - then you can easily manage the lifecycle.		Idealy, use a bucket just for logs - then you can easily manage the lifecycle.


			Lifecycle rules are crated in the bucket.


			= Storing logs in S3 vs Cloudwatch =


			:* The most '''cost effective''' may not be the least expensive.

			:* Consider the requirements and the effectiveness of the solution.

			::* Think S3 is just text files! - S3 Glacier is slow text files - If instant searching is a requirement these won't cut it.


			= Cloudwatch log subscription filters =

2026-06-08T00:34:47Z

Storing logs in S3

← Older revision		Revision as of 00:34, 8 June 2026
Line 104:		Line 104:

	:: An alternative to using tape.		:: An alternative to using tape.


			== Using S3 lifecycle with logs ==


			This approach helps reduce the stoage costs for logs automatically.

			:: Lifecycles can apply to all objects in a bucket or items with certain prefixes.

			::: Tags can also be used to assgin a file to a lifecycle.


			Idealy, use a bucket just for logs - then you can easily manage the lifecycle.

2026-06-08T00:13:32Z

S3 Glacier Deep Archive

← Older revision		Revision as of 00:13, 8 June 2026
Line 94:		Line 94:
	:: Great for low cost storage for compliance - I have it, I hope I don't need it.		:: Great for low cost storage for compliance - I have it, I hope I don't need it.

	== S3 Glacier Deep Archive ==		=== S3 Glacier Deep Archive ===

2026-06-08T00:13:20Z

S3 Glaicer Flexable

← Older revision		Revision as of 00:13, 8 June 2026
Line 81:		Line 81:
	:: Same Resillanacy and avalability.		:: Same Resillanacy and avalability.

	== S3 Glaicer Flexable ==		=== S3 Glaicer Flexable ===


Line 93:		Line 93:

	:: Great for low cost storage for compliance - I have it, I hope I don't need it.		:: Great for low cost storage for compliance - I have it, I hope I don't need it.


	== S3 Glacier Deep Archive ==		== S3 Glacier Deep Archive ==

2026-06-08T00:12:59Z

S3 Glaicer Flexable =

@@ Line 81: / Line 81: @@
 :: Same Resillanacy and avalability.
-== S3 Glaicer Flexable ===
+== S3 Glaicer Flexable ==
-:*
+::: Data access isn't instant.

2026-06-08T00:08:21Z

S3 has several storage tiers

← Older revision		Revision as of 00:08, 8 June 2026
Line 80:		Line 80:

	:: Same Resillanacy and avalability.		:: Same Resillanacy and avalability.


	== S3 Glaicer Flexable ===		== S3 Glaicer Flexable ===