MAD2-Backend-Jobs

 Questions related to backend jobs

1. SMTP (Simple Mail Transfer Protocol)?

SMTP is a communication protocol used to send emails over the Internet. It works by transferring email messages between mail servers or from a client (e.g., your email application) to a mail server.

SMTP operates on the Application Layer and uses a store-and-forward mechanism

Email message consists of a header (e.g., sender, recipient, subject) and body (the email content).

Using MailHog to Send Emails via SMTP

MailHog is a free email testing tool that acts as a fake SMTP server for testing emails locally. It's especially useful during development to test email functionality without sending real emails.

2.Caching

from flask_caching import Cache

cache = Cache()

from the above code cache instance is created in the Flask application using the Flask-Caching extension.

Flask_Caching= Is a Flask library

Cache = Is a class imported from the flask caching library

1.Cache data and improve the performance of Flask applications by storing frequently accessed data temporarily.

2.This creates an instance of the Cache class. This instance (cache) will be used to configure and interact

with the caching system in your Flask app

3.Caching stores computed data, results of API calls, or expensive database queries in

memory (or another storage backend) for faster retrieval later.

4.When a subsequent request asks for the same data, the cached data is returned instead of

recomputing it or re-fetching it from the database.

5.To use this cache instance in your Flask app, you need to initialize it with the

app and configure it with a caching backend.

cache = Cache(app) # Initialize cache with the Flask app

6.# Configure caching (e.g., using simple in-memory cache)

app.config['CACHE_TYPE'] = 'RedisCache' # Other types: 'SimpleCache', 'FileSystemCache', etc.

7.Caching is used mainly for expensive API calls, In your project caching is implemented

with GET requests(/Users and /profile of user)

8.Ex:

@app.route('/expensive') #Flask view function (or endpoint).

@cache.cached(timeout=60) # Cache the first time result of this route for 60 seconds def expensive_view(): # Simulate an expensive computation return "This is cached data."

When the /expensive route is accessed, Flask-Caching stores its output for 60 seconds.

If the same route is accessed within the next 60 seconds, the cached response is returned

without re-executing the view function.

timeout =This specifies how long the cached result should be stored, in seconds.

After this time cache will expire, and the function will execute again to generate a fresh result.

USES:

Improved Performance: Speeds up response times by avoiding redundant computations or database queries.

Reduced Server Load: Less work for the server to handle repeated requests for the same data

Flask-Caching supports multiple backends, including:

• SimpleCache (in-memory, for single-instance apps)
• RedisCache (requires a Redis server)
• FileSystemCache (stores cached data in files)
• Memcached (requires a Memcached server)

What is memoization?

Memoization is a way to cache a return value of a function based on its parameters

Caching is storing of data so that future requests for that data can be served faster. The data stored in a cache may be the result of an earlier computation or a copy of data stored elsewhere.

Memoization is a specific form of caching where the results of a function are stored based on its inputs.

It is typically used to optimize deterministic functions (functions that return the same result for the same inputs).

Scope:

• Used within a single program or function to optimize computational tasks.
• Focused on function-level optimization.
• Results are stored in a data structure like a dictionary

Usage:

• Commonly used in recursive algorithms (e.g., Fibonacci, dynamic programming).

Memoization is a type of caching that focuses on function calls and is usually implemented at the code level.

Caching is a broader concept used to store and retrieve any data efficiently, applicable across the entire application or system.

Memoization: "Store results of this function for reuse."

Caching: "Store any reusable data for faster access.

Where did you implement caching? Application Level Caching (In-Memory Caching) Where: Inside your application code, typically for function results, database queries, or API responses. @app.route('/users', methods=['GET'])---fourth executed @auth_required("token")----third executed @roles_required("admin")-------second executed @cache.cached(timeout=10)-----first executed This is application-level caching provided by Flask-Caching. Specifically,

this is view function output caching, where the entire response of the route (/users) is cached for the specified timeout (10 seconds).

Reduces database query overhead.

Improves response time for frequently accessed endpoints.

Lightens server load for endpoints with repetitive responses.

What is celery?

Celery is an open-source distributed task queue and asynchronous job scheduler that allows you to execute

tasks in the background or distribute them across multiple workers. It's commonly used in Python applications.

If your application has a long running task, such as processing some uploaded data or sending email, you don’t want to wait for it to finish during a request. Instead, use a task queue to send the necessary data to another process that will run the task in the background while the request returns immediately.

Client <--> Flask App <--> Redis (Message Broker) <--> Celery Workers (W1, W2, W3...) | | | |--> Request Task -->|--> Add Task to Redis Queue ----->|--> Process Task |<-- Response ------>|<--- Notify Completion -----------|<--- Update Status/Result

1.Client send requests and receive responses from Flaskapp

2.Flaskapp puts the requests in the database queue

3.Celery Application: Celery runs as a separate process and communicates with the

Flask app through a message broker (e.g., Redis).

4.Task Queue: Tasks are pushed into a message queue (broker) by the Flask app.

5.Workers: Celery workers listen to the queue, fetch tasks, execute them, and update the status in the broker or a results backend.

Tasks: Functions decorated with @celery.task.

Message Broker: Redis or RabbitMQ acts as a middleman for queuing tasks.

Workers: Processes that execute the queued tasks.

Results Backend: Tracks the results of completed tasks.

What are components of celery?

1. Task: A function decorated with Celery that will be executed asynchronously.
2. Broker: A message queue (e.g., RabbitMQ, Redis) that queues and distributes tasks.
3. Worker: A process that listens for tasks and executes them in the background.
4. Result Backend: Stores the results of completed tasks for later retrieval.
5. Scheduler (Celery Beat): A component to schedule periodic tasks.
6. Flower: A web-based monitoring tool to visualize Celery tasks and workers.
7. Task Queue: Organizes tasks and helps in managing task priorities (optional).
8. Client: The part of the application that triggers tasks and sends them to the broker.

These components work together to manage and execute background tasks efficiently in a distributed system.

Celery consisits of multiple brokers

Difference between memcached and redis cache?

Memcached and Redis are both popular in-memory data stores used for caching

Memcached:

• Memcached is a simple, key-value store.
• It only supports string-based values (e.g., integers, text, or serialized data).
  
• Memcached is purely an in-memory store. It doesn’t offer persistence features.
• When the server is restarted, all data in Memcached is lost.
  Memcached is very fast and lightweight because it focuses on a simpler set of features.

Redis:

• Redis supports a wide range of data structures including strings, lists, sets, hashes, sorted sets, bitmaps, hyperloglogs, and geospatial indexes.
• This allows Redis to be more versatile, supporting complex operations like sorting, ranking, and more, in addition to simple caching.
  
• Redis offers optional persistence, which means it can save data to disk periodically.
• Redis supports two types of persistence:
• Snapshotting (RDB): Saves data at regular intervals.
• Append-Only File (AOF): Logs every write operation, offering more durability.
  
  
• Redis is generally slightly slower than Memcached for simple operations due to its more complex feature set and persistence options.
  

Memcached: Ideal for simple, high-performance caching where you just need to cache raw data (e.g., database query results, sessions). It’s best when you need a fast, lightweight cache with no persistence.

Redis: Best for complex caching scenarios where you need advanced data structures, persistence, or features like pub/sub, real-time analytics, or message queuing. It’s suitable for high-performance caching, task queues, or managing more sophisticated data types.

Message Broker?

Message brokers can validate, store, route, and deliver messages to the appropriate destinations. They serve as intermediaries between other applications, allowing senders to issue messages without knowing where the receivers are, whether or not they are active

ex:RabbitMQ, Apache kafka ,Redis