Problem domain: Analytics

You define jobs in AWS Glue to accomplish the work that’s required to extract, transform, and load (ETL) data from a data source to a data target. You typically perform the following actions:

• For data store sources, you define a crawler to populate your AWS Glue Data Catalog with metadata table definitions. You point your crawler at a data store, and the crawler creates table definitions in the Data Catalog. For streaming sources, you manually define Data Catalog tables and specify data stream properties.
• In addition to table definitions, the AWS Glue Data Catalog contains other metadata…

Problem domain:
Asynchronous Communication in Microservice World

What does it offer?
Event ingestion from many sources(AWS services or Third party)
Event delivery (Targets can be AWS service or Third party)
Event security
Event access Authorization
Event consumption Error handling

Benefits:
Connect data from SaaS apps
EventBridge ingests data from supported SaaS applications and routes it to AWS service targets through native integration in the AWS management console.

Write less code
EventBridge makes it easy to connect applications together because you can ingest, filter and deliver events without writing custom code.

Easily build event-driven architectures

With EventBridge, your event targets don’t…

a) How to represent time in an universal format?
Answer: As a number
Explanation:
Epoch time — The Unix epoch (or Unix time or POSIX time or Unix timestamp) is the number of seconds that have elapsed since January 1, 1970.
Ex: 1606982441

In Nodejs,

let dateNow = Date.now();
console.log(dateNow) //1606984719575

So dateNow is a number and is in Unix Epoch.

b) How to convert time in number to specific date and time according to timezone?

let date = new Date(dateNow)
let temp = date.toLocaleDateString(undefined, {timeZone:’Asia/Kolkata’});
console.log(temp) // 3/12/2020

Important note: The response of date.toLocaleDateString is dependent on the Operating system or Platform.

In Windows, the output will be 3/12/2020 i.e dd-mm-yyyy In Linux…

• A producer is a user application that sends messages.
• A queue is a buffer that stores messages.
• A consumer is a user application that receives messages.

The core idea in the messaging model in RabbitMQ is that the producer never sends any messages directly to a queue. Actually, quite often the producer doesn’t even know if a message will be delivered to any queue at all.

Instead, the producer can only send messages to an exchange. An exchange is a very simple thing. On one side it receives messages from producers and the other side it pushes them to queues.

…

Very good blog to read about the life cycle of messages in SQS

http://pragmaticnotes.com/2017/12/19/lifecycle-of-a-message-in-amazon-sqs-a-detailed-coverage/#:~:text=In%20most%20cases%2C%20multiple%20consumers,a%20large%20number%20of%20messages.&text=There%20is%20no%20concept%20of,visibility%20timeout%20provided%20by%20SQS.

Important points:

1. Delivery of Messages can be delayed
2. Messages have an expiry
3. Dead letter queues if unable to process a message. Unable to process is identified by number of retries to read a message.
4. Once a message is being read by a consumer, this message shouldn’t be processed by another service at the same time. So when a message is being read, you can configure SQS so that same message wont be read by multiple consumers. This concept is called as Visibility Timeout.

It is an open OASIS and ISO standard lightweight, publish-subscribe network protocol that transports messages between devices.

Historically, the “MQ” in “MQTT” came from the IBM MQ.

Just like HTTP has a client and server, the MQTT protocol defines two types of network entities: a message broker and a number of clients. An MQTT broker is a server that receives all messages from the clients and then routes the messages to the appropriate destination clients. …

I have been trying to follow the below medium link to create PDF from html and I came to know some problems and solutions.

Problems:

1. My CSS were not getting applied on the generated PDF
2. Also I want to know whether my page has been loaded before I do the conversion.
3. I want to get all console.log messages to see if my page has any errors.

Solutions:

1. Use printBackground: true option in pdf generation api call.
2. Use Inline CSS in html document.
3. Use events on page to get console logs and loading status.

Code snippet:

const templateFile = fs.readFileSync(resumeTemplatePath, ‘utf8’);

const…

A bookmark of useful links

Memory Efficiency in Nodejs using Buffers and Streams

One important note: A pipe can have both ends.One end to read and other end to write. Remember that read speed and write speed has to be taken care while you program, otherwise if write is slower than read, your memory might get crashed as it has to wait for write to complete.

Only read as fast as you can write.

Understand the concept of Back pressure

https://medium.com/@jayphelps/backpressure-explained-the-flow-of-data-through-software-2350b3e77ce7

Max size of a buffer in Nodejs: https://stackoverflow.com/questions/8974375/whats-the-maximum-size-of-a-node-js-buffer#:~:text=Maximum%20length%20of%20a%20typed,2Gb%20%2D%201byte%20on%2064%2Dbit

Converting an Html view to PDF

Handlebars in Nodejs

Very famous Node library list

Using Nodemon in docker

https://medium.com/better-programming/docker-in-development-with-nodemon-d500366e74df