The Problem With Callbacks (A Quick Recap)

In the last article, we talked about callbacks - a function you pass to other functions so they can be called later. They work. But when you need multiple async operations to happen in sequence, the code starts nesting deeper and deeper.

getUser(function(err, user) {
  getProfile(user.id, function(err, profile) {
    getPosts(profile.username, function(err, posts) {
      // buried three levels deep
    });
  });
});

Hard to read. Hard to debug. Hard to maintain. This is the problem promises were built to solve.

A Promise Is a Future Value

Here's the mental model that makes promises click:

A promise is a placeholder for a value you don't have yet.

Think about ordering food at a restaurant. You place your order, and the waiter gives you a little buzzer. That buzzer is a promise - it doesn't have your food right now, but it represents a commitment that food is coming. You don't stand at the counter waiting. You go sit down, do other things, and when the buzzer goes off, you know your order is ready.

That's exactly what a JavaScript Promise does. When you kick off an async operation, you get a promise back immediately. You can say: "when this resolves, do this. If it fails, do that."

The Three States of a Promise

Every promise is always in one of three states:

1. Pending:
   Operation still in progress. Waiting for the buzzer.

2. Fulfilled:
   Completed successfully. You got your food.

3. Rejected:
   Something went wrong. The kitchen ran out.

A promise starts as pending. It then either fulfils or rejects - and once it does, it stays that way. A fulfilled promise can't become rejected. That's it, no going back.

Creating a Basic Promise

Here's what a promise looks like from the inside:

const myPromise = new Promise(function(resolve, reject) {
  const success = true;

  if (success) {
    resolve("It worked!");
  } else {
    reject("Something went wrong.");
  }
});

You create a promise by passing a function to new Promise(). That function receives two arguments: resolve and reject. You call resolve when things go well, and reject when they don't .

A more realistic example - wrapping setTimeout in a promise:

function wait(ms) {
  return new Promise(function(resolve) {
    setTimeout(resolve, ms);
  });
}

wait(2000).then(function() {
  console.log("2 seconds passed");
});

Instead of nesting a callback inside setTimeout, you get a promise back that you can chain cleanly off of.

Handling Success and Failure

You handle a fulfilled promise with .then(), and a rejected promise with .catch():

fetch("https://api.example.com/user")
  .then(function(response) {
    return response.json();
  })
  .then(function(data) {
    console.log(data);
  })
  .catch(function(error) {
    console.log("Something went wrong:", error);
  });

A few things to notice:

• .then() receives the resolved value

• .catch() receives the rejection reason

• One .catch() at the end handles errors from anywhere in the chain.

With callbacks, you had to manually check if (error) inside every single callback.
With promises, error handling is centralised. One .catch() covers the whole chain.

Promise Chaining

This is the main feature that made promises revolutionary. Each .then() returns a new promise, so you can chain them in a flat, readable sequence:

fetch("https://api.example.com/user")
  .then(response => response.json())
  .then(user => fetch("/api/posts/" + user.id))
  .then(response => response.json())
  .then(posts => console.log(posts))
  .catch(error => console.log(error));

Now compare the same logic, written both ways:

Callbacks:

getUser(function(err, user) {
  if (err) return log(err);
  getPosts(user.id,
    function(err, posts) {
      if (err) return log(err);
      display(posts);
    }
  );
});

Promises:

getUser()
  .then(user =>
    getPosts(user.id)
  )
  .then(posts =>
    display(posts)
  )
  .catch(err => log(err));

Same logic. Completely different readability. The promise version reads top to bottom like a sequence of steps. Flat code is genuinely easier to reason about - especially when you're debugging at midnight.

The Full Lifecycle

const promise = new Promise(function(resolve, reject) {
  // 1. Starts as "pending"
  setTimeout(function() {
    const worked = Math.random() > 0.5;

    if (worked) {
      resolve("Success!");   // 2a. Fulfilled
    } else {
      reject("Failed!");     // 2b. Rejected
    }
  }, 1000);
});

promise
  .then(value => console.log("Fulfilled:", value))
  .catch(reason => console.log("Rejected:", reason))
  .finally(() => console.log("Done either way"));

.finally() runs regardless of whether the promise is fulfilled or rejected. Handy for cleanup - hiding a loading spinner, closing a connection, that sort of thing.

A Few Things Worth Knowing

Promises are eager. The function you pass to new Promise() runs immediately - not lazily. The async part is what gets deferred.

.then() always returns a new promise. This is why chaining works. Each step waits for the previous one to resolve before running.

If you return a promise inside .then(), the chain waits for it. Returning fetch(...) inside a .then() means the next .then() doesn't run until that fetch completes.

Always handle rejections. It you forget .catch() and the promise rejects, you'll get a warning in Node.js - and potentially a crash. Never skip error handling.

Conclusion

Promises are not magic.
They're just a better way to manage the same callbacks you already know. The mental model - a future value, three states, .then() and .catch() - is what everything else builds on.

Want More?

Blog: https://blogs.kanishk.codes
Twitter: https://x.com/kanishk_fr
LinkedIn: https://linkedin.com/in/kanishk-chandna
Instagram: https://instagram.com/kanishk__fr

Functions Are Just Values

Before we even get to callbacks, there's one thing you need to internalise about JavaScript: functions are values.

That's it. That's the foundation of everything in this article.

In most languages, a function is a special construct - you define it, you call it, that's it. But in JavaScript, a function is just a value like a number or a string. You can store it in a variable, put it in an array, pass it to another function, or return it from a function.

const greet = function() {
  console.log("Hello!");
};

greet(); // "Hello!"

This is not just a fun quirk. It's what makes callbacks possible - and what makes JavaScript's async model work the way it does.

What Is a Callback Function?

A callback is simply a function you pass to another function, so that the other function can call it later.

That's the whole definition. Nothing more complicated than that.

function doSomething(callback) {
  console.log("Doing something...");
  callback();
}

function finished() {
  console.log("Done!");
}

doSomething(finished);
// "Doing something..."
// "Done!"

Notice you're passing finished - not finished(). You're passing the function itself, not the result of calling it. The doSomething function decides when to call it.

That "deciding when to call it" part is what makes callbacks useful.

Passing Functions as Arguments

Let's look at a few more examples before we get into async territory, because callbacks aren't only for async code.

Callbacks in array methods

You use callbacks all the time without thinking about it.

const numbers = [1, 2, 3, 4, 5];

const doubled = numbers.map(function(n) {
  return n * 2;
});

console.log(doubled); // [2, 4, 6, 8, 10]

The function you pass to .map() is a callback. You're telling .map(): "Here's what I want you to do with each element." .map() handles the looping - you just hand it the logic.

const evens = numbers.filter(n => n % 2 === 0);
// [2, 4]

numbers.forEach(n => console.log(n));
// 1, 2, 3, 4, 5

A custom example

function repeat(n, action) {
  for (let i = 0; i < n; i++) {
    action(i);
  }
}

repeat(3, function(i) {
  console.log("Step " + i);
});
// "Step 0"
// "Step 1"
// "Step 2"

This pattern - a function that takes another function and decides when to call it - is everywhere in JavaScript.

Why Callbacks Exist in Async Programming

Here's where callbacks really shine. JavaScript is single-threaded. That means it can only do one thing at a time. There's no parallel execution, no background threads.

So what happens when you need to do something that takes time - like fetching data from a server, reading a file, or waiting for a user to click something?

You can't just pause everything and wait. That would freeze the entire page. Instead, JavaScript says: "I'll start this task, move on to other things, and call you back when it's done."

setTimeout - the simplest async example

console.log("Start");

setTimeout(function() {
  console.log("This runs after 2 seconds");
}, 2000);

console.log("End");

What do you think the output is?

Start
End
This runs after 2 seconds

JavaScript doesn't pause at setTimeout. It registers the callback, starts a timer in the background, and moves on. When 2 seconds pass, it comes back and runs your callback. This is the event loop in action.

Callback Usage in Common Scenarios

Event listeners

Every time you attach a click handler, you're using a callback:

document.querySelector("button").addEventListener("click", function() {
  console.log("Button clicked!");
});

You're telling the browser: "when this button is clicked, call this function." You don't know when the click will happen - that's up to the user. The callback is the instruction you leave behind.

Fetching data - the old way

Before fetch() and promises, the standard was XMLHttpRequest. The (error, data)pattern it used - called error-first callbacks - was the convention in Node.js for years.\

function getData(url, callback) {
  const xhr = new XMLHttpRequest();
  xhr.open("GET", url);
  xhr.onload = function() {
    if (xhr.status === 200) {
      callback(null, xhr.responseText);
    } else {
      callback("Error: " + xhr.status, null);
    }
  };
  xhr.send();
}

getData("https://api.example.com/user", function(error, data) {
  if (error) {
    console.log("Something went wrong:", error);
    return;
  }
  console.log("Got data:", data);
});

Reading files in Node.js

const fs = require("fs");

fs.readFile("notes.txt", "utf8", function(error, content) {
  if (error) {
    console.log("Couldn't read file:", error);
    return;
  }
  console.log(content);
});

console.log("This runs before the file is read");

readfile starts the operation, moves on, and calls your callback when the file is ready.

The Basic Problem: Callback Nesting

Callbacks work well for one or two async operations. But things get messy fast when you need to chain multiple async tasks - where the result of one feeds into the next.

Imagine you need to: get a user's ID, use that ID to fetch their profile, then use the profile to fetch their posts. With callbacks:

getUser(function(error, user) {
  if (error) return console.log(error);

  getProfile(user.id, function(error, profile) {
    if (error) return console.log(error);

    getPosts(profile.username, function(error, posts) {
      if (error) return console.log(error);

      console.log(posts);
    });
  });
});

Each callback is nested inside the previous one. The code keeps drifting to the right. This is what people call callback hell - or the pyramid of doom, because of the shape it creates.

step1(function(err, result1) {
  step2(result1, function(err, result2) {
    step3(result2, function(err, result3) {
      step4(result3, function(err, result4) {
        step5(result4, function(err, result5) {
          // you get the idea
        });
      });
    });
  });
});

It's not just an aesthetic problem. It becomes genuinely hard to follow the logic flow, handle errors consistently, and refactor or reuse parts of the code.

Important: Callbacks are not bad. They're the building block. Once you understand them deeply, everything that comes after - Promises, async/await - will make a lot more sense, because they're all just cleaner ways of doing the same thing callbacks were doing.

The Mental Model

Here's a simple way to think about all of this:

A callback is an instruction you hand to someone else, telling them what to do once they're ready.

You're not standing there waiting. You're leaving a note. When the task is done, the note gets executed. Whether it's a button click, a file read, a network request - same idea. You say: "Do your thing, and when you're done, call this."

Conclusion

Callbacks exist because JavaScript needed a way to handle things that take time without blocking everything else. You've been using them already - every .map(), every addEventListener, every setTimeout. Now you know why it works the way it does.

Want More?

Blog: https://blogs.kanishk.codes
Twitter: https://x.com/kanishk_fr
LinkedIn: https://linkedin.com/in/kanishk-chandna
Instagram: https://instagram.com/kanishk__fr

Before we talk about polyfills, let's make sure we're on the same page about what string methods actually are.

In JavaScript, a string isn't just a bunch of characters; it's an object under the hood.
And like any object, it comes with built-in methods you can call on it. Things like .toUpperCase() , .trim(), .includes(), .slice - these are all methods that live on String.prototype.

const name = "kanishk";
console.log(name.toUpperCase()); // "KANISHK"
console.log(name.includes("ani"));  // true

Simple, right? You write a string, call a method, and get a result. But these methods didn't always exist. JavaScript evolves constantly, and each new ECMAScript spec adds new ones. That's exactly where polyfills come in.

Why Would You Even Write a Polyfill?

A polyfill is basically you saying: "Hey, this built-in method might not exist in older environments - so let me implement it myself, just in case."

Think about it. You're building a web app that needs to run in older browsers. You use String.prototype.includes() freely in your code, but then someone opens your app in an old browser that doesn't support it - boom, everything breaks.

A polyfill patches that gap:

if (!String.prototype.includes) {
  String.prototype.includes = function(search, start) {
    return this.indexOf(search, start) !== -1;
  };
}

Interview insight: When an interviewer asks you to implement String.prototype.repeat from scratch, they're not checking if you memorised the spec. They're checking if you understand what the method actually does at a logical level. Can you think through the steps? Can you handle edge cases?

Implementing Common String Utilities

Let's go through the most commonly asked ones, one by one.

String.prototype.repeat
Repeats a string n times. Single concept, but the edge cases are what matter.

String.prototype.myRepeat = function(count) {
  if (count < 0) throw new RangeError("Invalid count value");
  if (count === 0) return "";

  let result = "";
  for (let i = 0; i < count; i++) {
    result += this;
  }
  return result;
};

console.log("ab".myRepeat(3)); // "ababab"

The logic is dead simple - loop count times, keep appending the string. The important bit is handling edge cases: negative count throws an error, zero returns an empty string.

String.prototype.includes
Checks if a string contains another string. This one's almost trivially easy because indexOf does the heavy lifting.

String.prototype.myIncludes = function(searchStr, position = 0) {
  return this.indexOf(searchStr, position) !== -1;
};

The concept: indexOf Returns -1 when the substring isn't found, and any other number when it is. We just convert that to a boolean.

startsWith and endsWith
These check if a string starts or ends with a particular substring. The key insight is uses slice to cut out the exact portion you want to compare.

String.prototype.myStartsWith = function(searchStr, position = 0) {
  return this.slice(position, position + searchStr.length) === searchStr;
};

String.prototype.myEndsWith = function(searchStr, endPosition = this.length) {
  const start = endPosition - searchStr.length;
  return this.slice(start, endPosition) === searchStr;
};

trim, trimStart, trimEnd
These remove whitespace from a string. This is a good one to know because it introduces regex patterns that come up again and again.

String.prototype.myTrim = function() {
  return this.replace(/^\s+|\s+$/g, "");
};

String.prototype.myTrimStart = function() {
  return this.replace(/^\s+/, "");
};

String.prototype.myTrimEnd = function() {
  return this.replace(/\s+$/, "");
};

The pattern ^\s+ matches whitespace at the beginning, and \s+$ matches it at the end. The | in myTrim just means "either/or".

padStart and padEnd
These pad a string to a specific length. The tricky part is when padString is longer than one character - you need to repeat it and then trim it down to exactly the right length.

String.prototype.myPadStart = function(targetLength, padString = " ") {
  if (this.length >= targetLength) return String(this);

  const padNeeded = targetLength - this.length;
  let padding = padString.repeat(Math.ceil(padNeeded / padString.length));
  padding = padding.slice(0, padNeeded);

  return padding + this;
};

console.log("5".myPadStart(3, "0")); // "005"

Common Interview String Problems

These are the classics - ones that show up across interviews at startups, mid-size companies, and big tech alike.

Reverse a String

Sounds too simple. Interviewers often ask for the manual approach because they want to see if you can work with indices.

function reverseString(str) {
  let result = "";
  for (let i = str.length - 1; i >= 0; i--) {
    result += str[i];
  }
  return result;
}

console.log(reverseString("hello")); // "olleh"

Check for Palindrome

Always clean the string first. Real-world strings have spaces, punctuation, mixed case - always handle that.

function isPalindrome(str) {
  const clean = str.toLowerCase().replace(/[^a-z0-9]/g, "");
  const reversed = clean.split("").reverse().join("");
  return clean === reversed;
}

console.log(isPalindrome("A man, a plan, a canal: Panama")); // true

Count Character Occurrences

The pattern count[char] = (count[char] || 0) + 1 is a one-liner you should have memorised.

function charCount(str) {
  const count = {};
  for (let char of str) {
    count[char] = (count[char] || 0) + 1;
  }
  return count;
}

console.log(charCount("hello"));
// { h: 1, e: 1, l: 2, o: 1 }

Check if Two Strings are Anagrams

function isAnagram(str1, str2) {
  if (str1.length !== str2.length) return false;

  const sort = s => s.toLowerCase().split("").sort().join("");
  return sort(str1) === sort(str2);
}

console.log(isAnagram("listen", "silent")); // true

The sorting approach is clean and readable. There's also a frequency counter approach (using charCount), which is O(n) vs O(n log n). Worth knowing both.

Longest Substring Without Repeating Characters

A sliding window classic. You're maintaining a window of unique characters - whenever a repeat shows up, you shrink the window from the left.

function lengthOfLongestSubstring(s) {
  let map = new Map();
  let maxLen = 0;
  let left = 0;

  for (let right = 0; right < s.length; right++) {
    if (map.has(s[right])) {
      left = Math.max(left, map.get(s[right]) + 1);
    }
    map.set(s[right], right);
    maxLen = Math.max(maxLen, right - left + 1);
  }

  return maxLen;
}

console.log(lengthOfLongestSubstring("abcabcbb")); // 3

Why Understanding Built-in Behaviour Actually Matters

You can use .includes() every day without ever thinking about how it works. That's totally fine for shipping features. But in interviews, this deeper understanding separates good candidates from great ones.

When you implement a polyfill from scratch, you're forced to think about:

• What does this method return? A new string? A boolean? An index?

• What happens at the boundaries? Empty strings, negative indices, undefined inputs.

• What's the time complexity? Is my approach O(n)? O(n²)?

For example, most developers know str.includes(search) returns true or false.
But how many know it accepts a second argument - the position to start searching from? And what happens if you pass a RegExp as the search value? (It throws a TypeError.)

That kind of awareness doesn't come from memorising docs. It comes from implementing these things yourself, running them, breaking them, and fixing them.

Want More?

Blog: https://blogs.kanishk.codes/
Twitter: https://x.com/kanishk_fr/
LinkedIn: https://linkedin.com/in/kanishk-chandna/
Instagram: https://instagram.com/kanishk__fr/

Instead of writing random functions everywhere, OOP helps you group related data and behaviour together. Think of it as giving your code a little structure so it doesn't turn into chaos.
Let's break it down step by step.

What Object-Oriented Programming (OOP) menas

Object-Oriented Programming is a programming style where you structure your code around objects.

An object represents something with:

• Properties (data about the object)

• Methods (actions the object can perform)

For example, a Car object might have:
Properties:

• color

• brand

• speed

Methods:

• start()

• stop()

• accelerate()

So instead of thinking in isolated functions, OOP helps us think in terms of things and their behaviours.

A Simple Real-World Analogy

Imagine a car manufacturing company. Before producing cars, engineer create a blueprint.
The blueprint describes:

• color

• engine type

• number of doors

• features

But the blueprint itself is not a car.
It's just a template.

Using that blueprint, the company can produce many cars.
Each car is a separate object, but all follow the same design.

Programming works the same way.
In JavaScript:
Blueprint -> Class
Actual cars -> Objects

What is a Class in JavaScript?

A class is basically a blueprint for creating objects.
It defines:

• what properties will an object have

• what actions they can perform

Here is a simple class example.

class Car {
    brand = "Toyota";
}

This defines a blueprint called Car. But notice something important:
This code does not create a car yet.
It only defines how cars should look.

Creating Objects from a Class

To create an actual object, we use the new keyword.

class Car {
    brand = "Toyota";
}
const car1 = new Car();

Now car1 is an object created from the Car class.
You can access its properties like this:

console.log(car1.brand);

Output:

Toyota

You can create multiple objects from the same class.

const car1 = new Car();
const car2 = new Car();
const car3 = new Car();

All of them follow the same blueprint.
That's code reuseability, which is one of the biggest benefits of OOP.

The Constructor Method

Often, we want each object to have different values.
For example:
Car 1 -> Toyota
Car 2 -> BMW
Car 3 -> Tesla
This is where the constructor comes in.

A constructor is a special method that runs when a new object is created.

Example:

class Car {
    constructor(brand, color) {
        this.brand = brand;
        this.color = color;
    }
}

Now we can create different cars.

const car1 = new Car("Toyota", "Red");
const car2 = new Car("BMW", "Black");

console.log(car1.brand); //Toyota
console.log(car2.brand); //BMW

The constructor helps initialise the object with specific data.

Methods Inside a Class

Classes can also contain methods.
Methods are simply functions that belong to the object.

Example:

class Car {
  constructor(brand) {
    this.brand = brand;
  }

  start() {
    console.log(this.brand + " is starting...");
  }
}

Now create an object.

const car1 = new Car("Tesla");

car1.start();

Output:

Tesla is starting...

The method start() belongs to every object created from the Car class.

Basic Idea of Encapsulation

Encapsulation sounds fancy, but the idea is simple.
It means keeping related data and behaviour together inside a class. Instead of writing scattered variables and functions everywhere, we group them.

Example without OOP:

let name = "Rahul";
let age = 20;

function printDetails() {
  console.log(name, age);
}

Now imagine doing this for hundreds of students. That would be a mess.
Using a class keeps everything organised.

class Student {
  constructor(name, age) {
    this.name = name;
    this.age = age;
  }

  printDetails() {
    console.log(this.name + " is " + this.age + " years old.");
  }
}

Now the data and behaviour belong together. That's encapsulation in a beginner-friendly sense.

Assignment: Create Your Own Student Class

Create a class called Student.

Requirements:
Properties:

• name

• age

Method:

• printDetails()

Example solution:

class Student {
  constructor(name, age) {
    this.name = name;
    this.age = age;
  }

  printDetails() {
    console.log(this.name + " is " + this.age + " years old.");
  }
}

Create multiple students:

const student1 = new Student("Rahul", 20);
const student2 = new Student("Aisha", 22);

student1.printDetails();
student2.printDetails();

Output:

Rahul is 20 years old.
Aisha is 22 years old.

Conclusion

Object-Oriented Programming is simply a way to model real-world things in code.
Key ideas to remember:

• Class -> blueprint

• Object -> instance created from the blueprint

• Constructor -> initialises object data

• Methods -> actions the object can perform

• Encapsulation -> grouping data and behaviour together

You don't need to memorise everything right now.
Just remember the big idea:

OOP helps you write code that is easier to organise, reuse, and maintain.

And trust me, once your projects grow beyond a few files, you'll appreciate that structure.

Want More?

Blog: https://blogs.kanishk.codes/
Twitter: https://x.com/kanishk_fr/
LinkedIn: https://linkedin.com/in/kanishk-chandna/
Instagram: https://instagram.com/kanishk__fr/

What happens after you type a URL and press Enter?

You open your browser, type example.com, hit Enter…. and a website appears.
No explosions. No loading bars from the 2000s. Just a page.

Behind the scenes, though, your browser just ran a small orchestra of systems, all working together to turn into pixels.

This article explains that journey in simple language.

No memorisation. No deep engine internals. Just the flow.

What is a browser, really?

A browser is now “an app that opens a website”.
A browser is a translator and renderer.

• It fetches raw files from the internet (HTML, CSS, JavaScript)

• Understands what those files mean

• Converts them into a visual page you can see and interact with

Think of a browser as:
A factory that turns code into a living, clickable interface.

High-level parts of a browser

At a very high level, a browser has a few main pieces:

1. Under Interface - what you interact with

2. Browser Engine - the coordinator

3. Rendering Engine - turns code into visuals

4. Networking - fetches files from servers

5. JavaScript Engine - runs JavaScript code

6. Storage & Other Helpers - cache, cookies, history, etc.

You don’t need to remember this list.
Just know that each part has one job.

The User Interface (the obvious part)

This is the part you already know:

• Address bar

• Tabs

• Back / Forward buttons

• Refresh button

• Bookmark bar

Important thing to note:

The UI is separate from the web page.

Your browser’s tabs and buttons are not part of the website.
They are controlled by the browser itself.

Browser Engine vs Rendering Engine

This sounds scary. It’s not.

• Browser Engine:
  The manager. It connects the UI to everything else.

• Rendering Engine:
  The artist. It takes HTML + CSS and paints pixels on the screen.

So:

• Browser Engine says: “User typed a URL”

• Rendering Engine says: “Cool, give me the files, I’ll draw the page”

Different browsers use different engines (like Chromium or Gecko), but the flow stays the same, which is all that matters right now.

Networking: fetching the website

When you press Enter, the browser:

1. Figures out which server to talk to

2. Sends a request over the internet

3. Downloads files like:

   • HTML

   • CSS

   • JavaScript

   • Images

   • Fonts

At this point, the browser has raw text files, not a page.
Yet.

HTML parsing and DOM creation

The browser starts with HTML.

HTML is just text.
The browser needs structure.

So it parses the HTML

What does parsing mean?

Parsing means: Reading something piece by piece and understanding its structure.

Example:

<div>
    <h1>Hello</h1>
    <p>World</p>
</div>

The browser converts this into a tree structure called the DOM.

DOM (Document Object Model)

• The DOM is a tree

• Every HTML element becomes a node

• Parent elements contain child elements

Think of it like a family tree:

• is the parent

• and

  are children

This DOM is how the browser understands the page.

CSS parsing and CSSOM creation

Now CSS comes in.

CSS answers questions like:

• What colour?

• What size?

• Where should this element sit?

The browser parses CSS and builds another structure called the CSSOM.

CSSOM (CSS Object Model)

• Similar to the DOM

• Represents styles and rules

• Includes inheritance and priorities

So now the browser has:

• DOM → structure

• CSSOM → appearance

DOM + CSSOM = Render Tree

The browser now combines both.

Not every DOM element is visible.

• “display: none” elements don’t show

• Some nodes are just structural

The browser builds a Render Tree, which includes:

• Only visible elements

• Their final computed styles

This is the blueprint for drawing the page.

Layout (reflow): deciding positions

Now the browser asks:

• Where does each element go?

• How wide?

• How tall?

• Relative to what?

This step is called layout or reflow.

It’s like arranging furniture in a room before painting it.

Change the screen size?
Layout runs again.

Painting and display

Finally:

1. The browser paints pixels

   • Text

   • Colours

   • Borders

   • Images

2. Everything is sent to the screen

3. You see the website

This step can happen many times as:

• JavaScript runs

• Styles change

• You scroll or resize

The big picture

Here’s the full story:

1. User types a URL

2. Browser fetches files

3. HTML → DOM

4. CSS → CSSOM

5. DOM + CSSOM → Render Tree

6. Layout calculates positions

7. Painting draws pixels

8. Page appears

That’s it.

Conclusion

You don’t need to remember:

• All component names

• Every step in detail

• Engine differences

What matters is this mindset:
“A browser is a system that parses, understands, and renders content step-by-step.

Once that clicks, everything else becomes easier to learn later.

And congrats.
You now know more about browsers than most people who use them all day.

Want More?

Blog: https://blogs.kanishk.codes/
Twitter: https://x.com/kanishk_fr/
LinkedIn: https://linkedin.com/in/kanishk-chandna/
Instagram: https://instagram.com/kanishk__fr/

The challenge is not just blocking bots, but doing so without hurting real users. This article explains how to identify automated interactions, how detection works under the hood, and how to mitigate bots in a balanced, practical way.

Understanding the Problem First

Before talking about tools, it helps to understand a basic truth:
Bots behave differently from humans because they don’t have uncertainty.

Humans hesitate, scroll randomly, make mistakes, and act inconsistently. Bots are optimised for speed, repetition, and precision. Almost all bot-detection strategies rely on observing this difference.

Indicators of bot activity

Bot detection usually starts with recognising patterns that are statistically unlikely for humans.

Unnatural request patterns

A common red flag is request frequency. A human user might generate a few requests per second at most. Bots generate hundreds.

For example:

• A user browsing a product page might load the page, scroll, and click once.

• A bot scraping prices might hit the same endpoint repeatedly with no delay.

Consistency is also suspicious. Humans are irregular. Bots often operate on fixed intervals like every 200 ms.

Strange session behaviour

Looking at the entire session instead of individual requests reveals more clues:

• Sessions that last only a fraction of a second but perform meaningful actions.

• Sessions that never navigate between pages

• Sessions that repeatedly trigger the same endpoint (login, search, API calls)

A real user explores, whereas a bot targets.

Input behaviour that feels “too perfect”

Forms are a goldmine for detection:

• Humans take time to type

• Humans make mistakes and use backspace

• Humans don’t fill hidden fields

Bots often submit forms instantly, perfectly, and sometimes fill fields that are invisible to users. That difference alone can identify a large percentage of automated traffic.

How detection actually works

From simple rules to behavioural intelligence

Rule-based detection

This is the simplest layer and often the first one implemented.

Examples:

• Block more than 100 requests per minute from a single IP

• Throttle repeated failed login attempts

• Reject requests with malformed headers

This works well against basic bots but fails against more advanced automation using proxies or headless browsers.

Behavioral analysis

This is where detection becomes more reliable.
Instead of asking “Who is this user?”, the system asks:

• How long did they take before clicking?

• Did they scroll naturally?

• Do they move the mouse like humans?

• Is there variation in timing?

These signals are difficult to fake at scale. Even advanced bots struggle to replicate human randomness consistently.

Fingerprinting

Fingerprinting combines multiple browser and device attributes to create a probabilistic identity:

• User agent

• Screen size

• Time zone

• Rendering quirks (canvas, WebGL)

One attribute alone proves nothing. But when hundreds of sessions share an identical fingerprint, automation becomes the most reasonable explanation.

Third-party bot detection platforms

Services like Cloudflare or Google reCAPTCHA use machine learning models trained on massive datasets.

Instead of a binary decision, they assign a risk score indicating how likely a session is to be automated. This allows websites to react proportionally instead of blocking blindly.

Prevention and mitigation

Stopping bots without punishing real users

The goal is not to eliminate all bots. That’s unrealistic. The goal is to reduce harmful automation while preserving user experience.

Layered defenses

Effective protection is layered, not aggressive.

1. Monitor first
   Log suspicious behaviour before blocking. False positives are expensive.

2. Slow down suspicious traffic
   Rate limiting or artificial delays frustrate bots without affecting humans much.

3. Challenge only when needed

   CAPTCHAs should appear only when behaviour crosses a risk threshold.

4. Block repeat offenders
   IPs, ranges, or ASNs that repeatedly abuse the system can be denied entirely

Smart use of CAPTCHA

CAPTCHAs are useful, but overuse destroys user trust.

A good approach:

• No CAPTCHA for normal browsing

• CAPTCHA after multiple failed logins

• CAPTCHA on high-risk actions like password resets or bulk submissions

Modern invisible CAPTCHA systems score behaviour silently and only intervene when confidence is low.

Protecting APIs and sensitive endpoints

Bots love unprotected APIs.
Mitigation includes:

• Authentication tokens

• Strict request validation

• Short-lived credentials

• Rate limits per user, not just per IP

If a endpoint doesn’t need to be public, it shouldn’t be.

Risk-based decision making

Advanced systems assign each session a score instead of a verdict.

For example:

• Low risk: allow normally

• Medium risk: monitor or throttle

• High risk: challenge or block

This keeps defenses flexible and minimises harm to legitimate users.

Conclusion

Bot detection is not about catching robots. It’s about understanding behaviour.

Humans are inconsistent, slow, and imperfect. Bots are fast, repetitive, and precise. By observing these differences across requests, sessions and interactions, websites can reliably detect automation without relying on guesswork.

The best systems:

• Combine multiple weak signals instead of trusting one

• Respond proportianally instead of aggressively

• Prioritise user experience as much as security

In practice, effective bot mitigation feels invisible to real users and exhausting to bots.

Want more?

Blog: https://blogs.kanishk.codes/
Twitter: https://x.com/kanishk_fr/
LinkedIn: https://linkedin.com/in/kanishk-chandna/
Instagram: https://instagram.com/kanishk__fr/

The internet is just a massive pile of wires, routers, and vibes.
When data moves across it, things can go wrong constantly.

Packets can:

• Get lost

• Arrive late

• Arrive twice

• Arrive in the wrong order

So the internet uses protocols, which are basically agreements on how data should be sent.
Two of the most important ones are TCP and UDP.

They solve the same problem differently. Very differently.

What are TCP and UDP

Think of them as two delivery styles.

TCP

TCP is careful, slow, and responsible.
It checks everything. It waits for confirmation. It resends data if something goes wrong.

It’s like a courier service that makes you sign for the package.

UDP

UDP is fast, reckless, and emotionally unavailable.
It sends data and moves on. No confirmation. No retries.

It’s like shouting information through a megaphone and hoping someone hears it.

Key differences between TCP and UDP

When to use TCP

Use TCP when data accuracy matters more than speed.

Examples:

• Website

• APIs

• Login system

• Emails

• File uploads and downloads

• Payments

If missing or scrambled data would break things or cause panic, TCP is the correct choice.

Basically, anything involving:

• Money

• Accounts

• Persistence

• Trust

When to use UDP

Use UDP when speed matters more than perfection.

Examples:

• Live video streaming

• Online gaming

• Voice calls

• Live sports broadcasts

• DNS queries

If a packet gets lost, it’s usually better to skip it and move forward than wait.
No one wants their Zoom call to freeze just to recover a missed syllable.

Real-world analogy

• TCP = Phone call
  Both sides connect, talk in order, confirm messages, and hang up properly.

• UDP = Live announcement
  Information is broadcast instantly. If you miss it, tough luck.

Neither is “better”. They’re built for different jobs.

What is HTTP, and where does it fits

Now let’s talk about the protocol that causes the most confusion.
HTTP (Hypertext Transfer Protocol) is an application-level protocol.
Important sentence. Read it again.

HTTP defines:

• How requests are structured

• How responses look

• Methods like GET, POST, PUT, DELETE

• Status codes like 200, 404, 500

HTTP does not care how data physically travels.
That’s not its job.

Relationship between TCP and HTTP

Here’s the clean mental model:

• HTTP decides what to send

• TCP decides how to send it reliably

• IP decides where to send it

HTTP runs on top of TCP

TCP handles:

• Connection setup

• Packet ordering

• Retransmissions

• Data integrity

HTTP just assumes: “Cool, transport layer has my back.”

Why HTTP does not replace TCP

Because HTTP is not built to:

• Detect packet loss

• Handle retransmission

• Maintain low-level connections

• Guarantee order

If HTTP tried to do all that, it would basically reinvent TCP and do a worse job.
Layering exists so protocols don’t step on each other’s responsibilities.

Conclusion

• TCP and UDP are transport protocols

• TCP = reliable and ordered

• UDP = fast and lightweight

• HTTP is an application protocol

• HTTP relies on TCP to work properly

• They solve different problems at different layers

Once this clicks, networking stops feeling like black magic and starts feeling like organised bureaucracy.

Want more?

Blog: https://blogs.kanishk.codes/
Twitter: https://x.com/kanishk_fr/
LinkedIn: https://linkedin.com/in/kanishk-chandna/
Instagram: https://instagram.com/kanishk__fr/

Imagine shouting instructions across a crowded street.

Some words get lost.
Some arrive twice.
Some arrive out of order.
Some arrive when the other person wasn’t even listening yet.

That’s what raw data transfer looks like without rules. Computers are fast, and they need structure.

This is where TCP walks in like the boring but responsible adult.

What is TCP, and why is it needed?

TCP (Transmission Control Protocol) is a communication rulebook used on the Internet to send data reliably.

When you open a website, send a message, upload a file, or log in anywhere, TCP is usually working behind the scenes, making sure:

• Data reaches the right place

• Nothing important is lost

• Everything arrives in the correct order

• Both sides agree on what’s happening

Without TCP, the internet would feel like a glitchy group chat with a bad network.

Problems TCP is designed to solve

TCP exists because the Internet itself is unreliable.

It solves the problems like:

• Packet loss - data can disappear mid-way

• Out-of-order delivery - packets don’t always arrive in sequence

• Duplicate packets - sometimes data gets sent twice

• Unknown connection state - sender doesn’t know if receiver is ready

TCP doesn’t panic. It plans for all of this.

What is the TCP 3-Way Handshake?

Before any real data is sent, TCP makes sure both sides are ready.
This setup process is called the 3-Way Handshake.

Think of it like starting a phone call:

1. “Can you hear me?”

2. “Yes, I can hear you. Can you hear me?”

3. “Yes, we’re connected.”

Only after this does the actual conversation start.

Step-by-step: SYN, SYN-ACK, ACK

SYN (Synchronise)

The client says:
“Hey server, I want to talk. Are you available?”

This message contains a sequence number, which is basically:
“This is where I’m starting from.”

SYN-ACK (Synchronise + Acknoledge)

The server replies:
“I hear you. I’m ready. And I got your starting number.”

The server also sends its own sequence number.
Now both sides know:

• The order exists

• The other is listening

• Where counting should start

ACK (Acknowledge)

The client responds:
“Cool, I got your message too.”

Handshake complete. Connection established. Data can flow.

How data transfer looks in TCP

Once the connection is open, TCP starts sending data in chunks called segments.

Each segment includes:

• Sequence number - where this piece fits

• Acknowledgement number - what has been received so far.

The receiver keeps saying:
“I got data up to number X.”

This constant feedback loop keeps everything synchronised.

How TCP ensures reliability, order, and correctness

1. Reliability

   If a packet goes missing, TCP notices.
   No acknowledgement? TCP resends it.

   Simple and effective.

2. Order

   Packets can arrive in any order.
   TCP rearranges them using sequence numbers before handing data to the application.

   So your video doesn’t play sentence 5 before sentence 2.

3. Correctness

   TCP checks for corrupted data.
   If something looks wrong, it gets reset.

   Your files arrive intact, not haunted.

How TCP handles packet loss

TCP assumes failure is normal.
If a packet isn’t acknowledged within a certain time:

• It is retransmitted

• Sending speed may be reduced

• Network congestion is assumed, not ignored

TCP doesn’t rage. It adapts.

How TCP connection is closed

Ending a TCP connection is also polite. No ghosting.

FIN and ACK

1. One side sends FIN
   “I’m done sending data.”

2. Other side sends ACK
   “Got it.”

3. Other side sends its own FIN
   “I’m done too.”

4. Final ACK
   “Connection closed.”

Everyone agrees. No loose ends.

Conclusion

TCP is not flashy. It doesn’t try to be cool.
It:

• Establishes trust before talking

• Keeps track of every message

• Fixes mistakes silently

• Ends conversations cleanly

That’s why the internet mostly works, even when everything else feels broken.

And yes, all of this happens every time you open a website.
In milliseconds. Without asking for applause.

Want more?

Blog: https://blogs.kanishk.codes/
Twitter: https://x.com/kanishk_fr/
LinkedIn: https://linkedin.com/in/kanishk-chandna/
Instagram: https://instagram.com/kanishk__fr/

How the Internet Reaches Your Home or Office

When you open a website, the data does not teleport from a server to your laptop.

It flows through:

• Your local network

• Your internet service provider

• Multiple networks across the internet

• And finally reaches a server somewhere else

Between you and the internet sit several network devices, each with a very specific role.

Understanding these roles makes system design, debugging, and production deployments far less mysterious.

What Is a Modem?

Responsibility: Connect your local network to your Internet Service Provider (ISP).

A model is a device that communicates directly with your ISP using cable, fiber, or DSL. Your home network cannot speak the ISP’s language on its own. The modem translates between the two.

Think of the modem as:
The translator between your house and the outside world.

Important clarification:

• A modem does not manage devices

• A modem does not route traffic

• A modem only establishes the internet connection

If the modem is down, you have zero internet. No amount of restarting your router will save you.

What Is a Router?

Responsibility: Direct traffic between devices and networks.

A router decides where packets should go.

Inside your home and office:

• Your laptop

• Your phone

• Your TV

• Your smart toaster

All of them share one internet connection. The router figures out:

• Which device made which request

• Where the response should go

• How traffic moves between local devices and the internet

Modem vs Router

• A modem connects you to the internet

• The router manages traffic within and out of your network

Hub vs Switch: Working of Local Networks

Hub (Old and Mostly Useless)

Responsibility: Broadcast everything to everyone.

A hub sends incoming data to all connected devices, whether they asked for it or not.
You can think of a hub as someone shouting messages in a crowded room.

Problems:

• Massive inefficiency

• Security nightmare

• Practically extinct for good reasons

Switch (What We Actually Use)

Responsibility: Send data only to the intended device.

A switch learns which device is connected to which port and forwards data intelligently.

Why switches matter:

• Faster local communication

• Better security

• Foundation of modern LANs

What Is a Firewall?

Responsibility: Decide what traffic is allowed and what is blocked.

A firewall sits between networks and enforces security rules:

• Which ports are open

• Which IPs are allowed

• Which traffic is suspicious

Firewalls exist at multiple levels:

• Hardware firewalls

• Software firewalls

• Cloud firewalls

For backend engineers, this matters because:

• A service can be “running” but unreachable

• Ports can be blocked silently

• Misconfigured firewalls cause legendary debugging sessions

What Is a Load Balancer?

Responsibility: Distribute traffic across multiple servers.

When one server is now enough, you add more. But users still need one entry point.
That’s the job of a load balancer.

Load balancers:

• Improve performance

• Increase reliability

• Enable horizontal scaling

From a backend perspective:

• Your API instances are stateless

• The load balancer decides where each request goes

• If one instance dies, traffic shifts automatically

This is how real production systems survive traffic without breaking.

How All These Devices Work Together

A simplified real-world flow looks like this:

1. Your device sends a request

2. Switch forwards it locally

3. The router decides it’s going outside

4. Firewall checks if it’s allowed

5. The modem sends it to the ISP

6. Internet happens

7. On the server side, a load balancer receives it

8. The load balancer forwards it to a backend service

Each device does one job, and does it well. This separation of responsibility is not accidental. It’s how networks scale and stay debuggable.

Why Software Engineers Should Care

Even if you never touch hardware:

• APIs depend on routers, firewalls, and load balancers

• “Works on localhost” dies in production because of networking

• Cloud infrastructure is just these devices abstracted

Understanding network devices helps you:

• Debug outages faster

• Design scalable systems

• Communicate better with the infra and DevOps teams

• Stop blaming “the internet” for everything

Conclusion

Network devices are not magical boxes. They are specialised tools solving very specific problems.

Once you understand who does what, networking stops being scary and starts being predictable.

And predictable systems are the only ones worth building.

Want more?

Blog: https://blogs.kanishk.codes/
Twitter: https://x.com/kanishk_fr/
LinkedIn: https://linkedin.com/in/kanishk-chandna/
Instagram: https://instagram.com/kanishk__fr/

What is a Server?

In very simple terms, a server is a machine that stays online 24×7 and runs a program or a website that provides data, resources, and services to other computers (also known as clients) over the network using the client-server model, where the client requests and the server responds.

When you open a website, your browser is the client. The machine hosting that website is the server. The browser sends a request like “give me this page”, and the server replies with the data needed to show it.

That’s the entire internet in one sentence.

How Do Clients and Servers Talk?

Clients and servers don’t magically understand each other. They communicate by sending structured messages over the network.

Your browser does this quietly in the background. You type a URL, hit enter, and things just work.

But as a programmer, we often want to:

• Test a server

• Talk to an API

• See what a server actually returns

• Debug things when they break

For that, we need a way to send messages without a browser.
That’s where a cURL comes in.

What is cURL?

cURL (Client URL) is a command-line tool that lets you send requests to a server directly from the terminal and see the response.

You give cURL a URL, and it:

• Sends a request to the server

• Receives the response

• Shows it to you

Under the hood, it supports many protocols and advanced features, but you do not need to care about any of that right now.

Why Programmers Need cURL

cURL is useful because it removes the UI layer and shows you exactly what’s happening between a client and a server.

Programmers use cURL to:

• Test APIs before writing frontend code

• Check if a server is responding correctly

• Debug backend issues

• Automate simple network requests

• Learn how HTTP actually works

If you work with backend systems, APIs, or DevOps even a little, cURL becomes unavoidable.

Your first cURL Command

Let’s start with the simplest possible example.

curl https://example.com/

This command:

• Sends a request to example.com

• Fetches the response

• Prints it in the terminal

If the URL returns HTML, you’ll see HTML.
If it returns JSON, you’ll see JSON.
If it returns plain text, you’ll see plain text.

It just shows you what the server sends back.

Using cURL to Talk to APIs

APIs are just servers that expect structured requests and return structured responses, usually in JSON.

curl https://api.github.com

This sends a simple request and returns JSON data describing the API.

You didn’t open a browser.
You didn’t write any JavaScript.
You talked directly to the API.

This is why cURL is often the first tool used when working with APIs.

GET and POST Requests

For now, we’ll just focus on two request types:

• GET: asking the server for data

• POST: sending data to the server

A basic GET request is what you’ve already been doing.
POST requests come later, once you’re comfortable reading responses.

Common Mistakes Beginners Make with cURL

Some very normal mistakes:

• Trying to learn every flag on day one

• Copy-pasting complex commands without understanding them

• Panicking when the output looks “ugly”

• Assuming cURL is only for backend developers

cURL output is waw by design. It’s now meant to be pretty. It’s meant to be honest.
If you understand what you sent and what you got back, you’re already doing it right.

Conclusion

cURL is not a fancy tool. It’s a fundamental one.

Once you’re comfortable using cURL:

• APIs feel less intimidating

• Backend systems make more sense

• Debugging becomes easier

• You stop guessing and start verifying

You don’t need to master cURL to use it effectively. You just need to understand that it’s a direct line between you and a server.

Want more?

Blog: https://blogs.kanishk.codes/
Twitter: https://x.com/kanishk_fr/
LinkedIn: https://linkedin.com/in/kanishk-chandna/
Instagram: https://instagram.com/kanishk__fr/

When you type “google.com” into a browser, your machine has absolutely no idea where that is. Networks do not route packets based on vibes or brand recognition. They route packets using IP addresses. DNS exists to translate human-readable domain names into machine-readable IP addresses.

That translation process is called name resolution. It exists because nobody wants to memorise 142.250.195.46 just to check their email.

At a high level, DNS answers one question:
“What IP address should I talk to for this domain?”

dig: Your X-Ray for DNS

dig stands for Domain Information Groper, which sounds like it was named during a very different era of the internet.

dig is a diagnostic tool. You don’t use it to browse the web. You use it to inspect how DNS works, debug weird DNS issues, and understand who is responsible for answering what.

If DNS were a bureaucracy, dig would be the form that lets you see who approved what and why.

DNS Resolution Is Layered

DNS resolution does not immediately provide the answer. It moves through layers.
Root → TLD → Authoritative
Each layer knows just enough to point you to the next one. Nobody knows everything. This is intentional and very important for scalability.

“dig . NS” - Root Name Servers

When you run “dig . NS”, you’re basically asking:
“Who is responsible for the root of the DNS system?”

The answer is a list of root name servers. These servers do not know where “google.com” lives. What they do know is who manages “.com”, “.org”, “.in”, and every other top-level domain.

You can think of root servers as the front desk. They don’t know where every employee sits. They just know which department to send you to.

“dig com NS” - TLD Name Servers

Now you are asking the “.com” TLD servers:
“Who is authoritative for domains ending in .com?”

These servers still don’t know Google’s IP address. They only know which authoritative name servers are responsible for “google.com”.

Yes, DNS is a chain of directions.

“dig google.com NS” - Authoritative Name Servers

This tells you which name servers are authoritative for google.com.

Authoritative name servers are the final authority. They hold the actual DNS records - A, AAAA, MX, TXT, CNAME, etc.

“dig google.com” - The Full Resolution

This returns an A record (IPv4) and possibly an AAAA record (IPv6). That’s the IP address your browser will connect to.

Behind the scenes, this is what actually happened:

1. Your system asked a recursive resolver (usually your ISP).

2. The resolver asked a root server.

3. Root pointed to .com.

4. .com pointed to Google’s authoritative servers.

5. Authoritative servers returned the IP.

6. The resolver cached the result.

7. Your browser finally made the HTTP request.

All of this happened in milliseconds. The internet is held together by duct tape, caching, and a lot of optimism.

What NS Records Actually Represent

NS records define responsibility.
They answer the question: “Who is allowed to speak with authority for this domain?”

Without NS records:

• Delegation breaks

• Caching becomes dangerous

• DNS becomes a single giant failure point

NS records are the reason DNS scales globally without collapsing under its own weight.

Recursive Resolvers - The Middleman

Your browser does not walk the DNS hierarchy every time. That would be painfully slow.

Recursive resolvers:

• Do the walking for you

• Cache results

• Respect TTLs

• Save the internet from imploding

Every time you “dig”, you’re basically peeking behind the curtain to see what your resolver usually does quietly.

From “dig” Output to a Real Browser Request

When DNS resolution finishes, the browser finally gets an IP address. Only then does HTTP even enter the conversation.

No DNS resolution → No TCP connection → No HTTPS → No website.

So yes, before your browser loads a single pixel, DNS has already done a full distributed lookup across the globe.

Conclusion

DNS looks simple until you actually look at it. Then you realise it’s a carefully layered, globally distributed system that somehow works every day without drama.

Understanding DNS resolution is not about memorising record types. It’s about understanding delegation, trust, and caching at internet scale.

And now, when someone says “it’s a DNS issue”, you’ll know whether they actually mean it.

Want more?

Blog: https://blogs.kanishk.codes/
Twitter: https://x.com/kanishk_fr/
LinkedIn: https://linkedin.com/in/kanishk-chandna/
Instagram: https://instagram.com/kanishk__fr/

Have you ever wondered how a browser knows where exactly a website lives? Leave the browser for a second. I’m pretty sure that if you’re reading this article, you must have thought about how and where these domains are stored and how they work behind the scenes.

So, in this article, I’ll take you through the entirety of the working of DNS (Domain Name System) in a very beginner-friendly way. We’ll go through What is DNS? Why is it needed? What are DNS records and their types?

Need for Domain Names

As you must be knowing that every device connected to the internet has its own unique address, known as an IP (Internet Protocol) address, which is a unique numerical label. This enables every device to communicate and locate each other for sending and receiving data over the internet.

As there are billions of devices on the internet, humans cannot remember long numerical labels. For our convenience, we have developed domain names, which are human-friendly, unique web addresses that identify a website or server on the internet, acting as an easy-to-remember shortcut for complex numerical IP addresses. This makes navigation simple and easy.

What is DNS?

You can think of DNS as the internet’s phonebook, which maintains all the domain name records, and it translates human-friendly domain names (like www.google.com) into machine-readable numeric IP addresses (like 142.250.186.42) that computers use to find and connect, allowing users to access websites by typing simple names instead of long numbers.

What are DNS records, and why are they needed?

So far, DNS sounds simple, you type a domain name, and somehow you end up at the right server.

But DNS doesn’t work with just one kind of information. Different parts of the internet need different answers. Browsers, mail servers, verification systems, and load balancers all ask DNS different questions.

DNS records are how a domain answers those questions.

A DNS record is a structured instruction stored inside a domain’s DNS zone. Each record tells the internet what to do with a specific kind of request related to that domain.

One record might say, “send website traffic here.”
Another might say, “email for this domain goes there.”
Another might exist only to prove ownership or configure security.

You can think of DNS records as a rulebook for a domain.
When a request comes in, DNS checks the rulebook and responds based on the type of question being asked.

Types of DNS records

NS Record

An NS (Name Server) record in DNS specifies which DNS servers are authoritative for a domain, basically telling the Internet where to find all other DNS records (like A, MX, CNAME) for that domain, ensuring correct IP resolution and reliable service through primary/secondary servers.

If DNS were a help desk, NS records tell you which desk handles questions for this domain.

You must be wondering why it exists. So, DNS is distributed. There isn’t one global server that knows everything. Instead, responsibility is delegated in the following steps:

1. The root servers know which name servers handle “.com”

2. “.com” servers know which name servers handle “example.com”

3. Those name servers finally know the actual records

NS records are how this delegation works.

A Record

An A Record maps a domain name to an IPv4 address.
You can also think of it as - if a domain name is a contact name, and an A record is the phone number saved in it.

When you type “example.com” into a browser, DNS is asked: “What IP address should I connect to for this domain?”
The A record is the answer to that question.

If DNS were a phonebook, A records would be the actual numbers people dial.

AAAA Record

An AAAA record maps a domain name to an IPv6 address.
The only difference between an A and AAAA record is that A handles IPv4 addresses and AAAA does the same for IPv6 addresses.

Pv4 has a limited number of addresses, and the Internet outgrew it.
IPv6 exists to solve that problem with a much larger address space.

CNAME Record

A CNAME record makes one domain name an alias of another domain name.
If an A record is a phone number, a CNAME record is a contact shortcut that says, “use whatever number this other contact has.”

Basically, instead of storing an IP address, a CNAME points to another hostname, and DNS keeps resolving until it reaches an A or AAAA record.

For example, when a browser asks for “www.example.com”, DNS might respond: “That’s a CNAME for example.com.”
DNS then look up “example.com” and returns its A or AAAA record.

MX Record

An MX (Mail Exchange) record tells the Internet where emails for a domain should be delivered.

This is what actually happens:
When someone sends an email to “hello@example.com”, their mail server asks DNS: “Which server handles email for example.com?”
DNS answers using MX records.

Those MX records point to mail servers, not IP addresses directly.

TXT Record

A TXT record stores text-based instructions or verification data for a domain.

TXT records are how a domain proves things about itself and publishes rules that the internet follows.

Reason why it exists: Some services do not need a destination, like a website or email server.
They need confirmation, configuration, or policy.

How All DNS Records Work Together

A single domain name isn’t just a website. It’s an entry point for multiple services, and DNS decides how each request should be handled.

Here’s how it comes together.

When someone types your domain into a browser, DNS uses A or AAAA records to translate that name into an IP address. That’s how web traffic reaches your server.

When someone sends an email to your domain, DNS doesn’t care about those A records. Instead, it checks MX records to find out which mail servers are responsible for receiving email.

If you want multiple names to point to the same place, CNAME record steps in. They let one hostname act as an alias for another, so you don’t have to duplicate configuration.

Behind the scenes, NS records decide who is allowed to answer these questions in the first place. They delegate authority to specific DNS servers and make the entire system scalable.

Through all of this, TXT records quietly publish rules, proof, and security policies.
They help verify ownership, protect email, and allow services to trust your domain.

Every website you visit relies on this cooperation. You just don’t notice it, because when DNS is doing its job right, it stays invisible.

And that’s exactly the point.

Want more?

Blog: https://blogs.kanishk.codes/

Connect with me

Twitter: https://x.com/kanishk_fr/
LinkedIn: https://linkedin.com/in/kanishk-chandna/
Instagram: https://instagram.com/kanishk__fr/

You run a few commands, things get saved, history appears, mistakes disappear, and somehow your code survives bad decisions. But Git isn’t magic. It’s just extremely well-organised.

Once you understand what Git is actually doing under the hood, a lot of confusion disappears. Commands stop feeling random, and Git starts behaving predictably.

Let’s open the box.

How Git Works Internally

At its core, Git is not a file tracker.
It’s a content tracker.

Git doesn’t think in terms of “files changing line by line.” Instead, it stores snapshots of content, identified by hashes. Every time you commit, Git saves a snapshot of how your project looked at that moment and links it to previous snapshots.

Think of Git as:

• A database

• Filled with objects

• Connected by references

• Stored locally on your machine

Everything Git needs lives inside one place.
That place is the .git folder.

Understanding the .git Folder

When you run “git init”, git creates a hidden folder called “.git”.

This folder is the entire brain of Git.

Your project files live outside it.
Your project’s memory lives inside it.

The “.git” folder contains:

• The complete history of your project

• All commits you’ve ever made

• Information about branches

• References like HEAD

• Git’s internal database of objects

If you delete “.git”, your project becomes a normal folder again. No history. No version control.

That’s why Git exists as a separate system layered on top of your files, not mixed into them.

Git Objects: The Building Blocks

Git stores everything as objects. There are three core ones you actually need to understand.
No more. No less.

Blob: The content

A blob stores the content of a file.

Important detail:
Git does not store filenames in blobs. Only content.

If two files have identical content, Git stores them once and reuses the blob. Efficient and slightly smug about it.

Tree: The Structure

A tree represents a directory.

It maps:

• Filenames → blobs

• Folder names → other trees

In simple terms:

• Blobs hold what the file contains

• Trees hold where files live

Commit: The Snapshot

A commit ties everything together.

A commit contains:

• A reference to a tree (project structure)

• A reference to its parent commit

• Author info

• Timestamps

• Commit message

A commit doesn’t store file changes.
It stores a pointer to a complete snapshot.

That’s why Git can move backwards and forward in history so easily.

What Actually Happens During “git add”

This is where most beginners get confused, so let’s slow down.

When you modify a file:

• Git notices the change

• But it doesn’t store it yet

When you run “git add”:

1. Git takes the file’s content

2. Converts it into a blob

3. Stores it inside the “.git” object database

4. Updates the staging area to point to that blob

Nothing is committed yet.
You’re just telling Git: “This version matters.”

The staging area is not a waiting room.
It’s a preview of the next snapshot.

What Actually Happens During “git commit”

When you run “git commit”:

1. Git takes everything in the staging area

2. Builds a tree representing your project structure

3. Creates a commit object

4. Links it to the previous commit

5. Moves HEAD to point to this new commit

That’s it.
No different files.
No line-by-line patchwork.
Just clean snapshots connected in a chain.

How Git Tracks Changes Without Losing Its Mind

Here’s the clever part.
Git uses cryptographic hashes (SHA-1 or SHA-256 in newer versions).

Every object:

• Blob

• Tree

• Commit

Is identified by a hash derived from its content.

This means:

• If the content changes, the hash changes

• If the hash changes, Git knows something changed

• If the hash doesn’t match, corruption is detected

This gives Git:

• Data Integrity

• Deduplication

• Trust in history

You literally cannot alter a past commit without Git knowing.
Which is comforting. And mildly intimidating.

The Mental Model That Makes Git Click

Stop thinking of Git as a command-line tool.
Think of it as:

• A local database

• Of immutable objects

• Connected by references

• That you move around with commands

Commands don’t do magic.
They just move pointers and create objects.

Once you see Git this way:

• “git add” makes sense

• “git commit” makes sense

• Branches make sense

• Even rebasing starts to behave

Eventually.

Conclusion

You don’t need to memorise Git internals to use Git well.

But understanding what’s happening behind the scenes turns Git from something you fear into something you trust.

Git isn’t trying to confuse you.
It’s just very honest about remembering everything.

And sometimes, that honestly feels personal.

Git is one of those things every developer is expected to know, but rarely taught properly. This article focuses on understanding Git before memorising commands, so you can actually use it with confidence.

What is Git?

Git is a distributed version control system.
That sounds heavy, so let’s simplify it.

Git is a tool that:

• Tracks changes in your code over time

• Lets you save checkpoints of your work

• Helps multiple people work on the same project without overwriting each other

Think of Git as a timeline for your code. Instead of copying entire folders, Git remembers what changed, when it changed, and who changed it.

The “distributed” part means every developer has a full copy of the project history on their own machine.
You don’t need the internet or a central server just to track changes locally.

Why is Git used?

Git exists to solve problems that appear the moment a project grows beyond one person or one day of work.

Here’s what Git helps with:

1. Tracking History
   You can see exactly how your project evolved over time and revert to an older version if something breaks.

2. Safe Experimentation
   You can try new ideas without fear. If it fails, you can discard the changes without damaging the working code.

3. Collaboration
   Multiple developers can work on the same codebase at the same time without overwriting each other’s work.

4. Accountability
   Git shows who made which change and why. This is incredibly useful for debugging and teamwork.

In short, Git replaces guessing, copy-pasting, and panic with structure and confidence.

Git Basics and Core Terminologies

Before touching commands, it’s important to understand Git’s mental model.

Repository (Repo)

A repository is a project tracked by Git.
It contains your files and the complete history of changes.

You can think of it as a folder with memory.

Commit

A commit is a saved snapshot of your project at a specific point in time.

Each commit:

• Has a unique ID

• Contains the changes made since the last commit

• Usually includes a short message describing what changed

Good commits are small, focused, and meaningful.

Branch

A Branch is an independent line of development.

Branches allow you to:

• Work on new features safely

• Fix bugs without touching the main code.

• Experiment without breaking anything

The default branch is usually called “main” or “master”.

HEAD

HEAD is simply a pointer to where you currently are in the project history.

If you switch branches or move to an older commit, HEAD moves with you.

You rarely control HEAD directly, but understanding it helps Git make sense.

Common Git Commands

These are the core commands every beginner should know.

git init
Initialises a new Git repository in the current folder.
This tells Git: “Start tracking this project”.

git status
Shows the current state of your working directory.
It tells you:

• Which files are modified

• Which files are staged

• What Git is waiting for you to do next

git add
Stages files for the next commit.
“git add filename” or “git add . ”
This does not save changes. It prepares them.

git commit
Creates a snapshot of the staged changes.
“git commit -m “Add user login feature” “

git log
Shows the commit history.
You’ll see:

• Commit IDs

• Authors

• Dates

• Messages

This is your project’s memory.

A Basic Git Workflow

Here’s how a typical solo developer workflow looks:

1. Create a project folder

2. Run “git init”

3. Write some code

4. Check changes with “git status”

5. Stage changes using “git add”

6. Save progress using “git commit”

7. Repeat as the project grows

That’s it. Everything else builds on this.

Conclusion

Git isn’t hard. It’s just often introduced badly.

You don’t need to memorise every command or understand Git internals on day one. Focus on:

• Tracking changes

• Writing good commits

• Practicing regularly

Once the basic click, Git stops feeling scary and starts feeling like a safety net.

And that’s exactly what it’s meant to be.

You manage to share those files through email, and you, along with your friend, start working on the same project. After a few weeks, your friend sends you the updated code files via email, and now you have two versions of the same software. One, which you were working on and the second, of your friend’s.

Here, the actual problem arises: how to determine the exact changes you and your friend have made and which one to keep.

At this point, things get messy fast.

You open your friend’s email attachment and stare at a folder full of files that look exactly like yours. Same filenames. Same structure. No obvious labels screaming “THIS IS WHAT I CHANGED.” Now comes the fun part, which is comparing.

You start opening files one by one, scrolling line by line, trying to remember whether that function was yours or theirs. Maybe you copy some parts from your version, paste some from theirs, and hope nothing breaks. Sometimes it works. Sometimes the app refuses to run, and you have no idea why, because you just Frankenstein’d two codebases together at 2 AM.

And this is still the best-case scenario.

Now imagine both of you changed the same file. Maybe even the same function. Maybe in slightly different ways. Who wins? Your logic or theirs? Do you manually merge it? Do you trust memory? Do you flip a coin and move on?

There’s also no safety net. If you mess up while merging, the original versions are gone unless you made backups. And if you did make backups, they probably look like:

• project_old

• project_old_2

• project_old_2_fixed

• project_latest_final

Congrats, you’ve invented versioning. Poorly.

The real problem here isn’t email or distance or even collaboration. It’s the complete lack of history. You don’t know what changed, when it changed, or why it changed. You don’t know who introduced the bug. You don’t know which version was actually working last. All you have are folders and vibes.

As more people join the project, this approach collapses entirely. Files get overwritten. Changes get lost. Someone forgets to send an email. Someone edits an outdated copy. Suddenly, half the team is building features on code that no longer exists.

This is the moment where people realised they weren’t just sharing files anymore. They were sharing responsibility, decisions, and mistakes. And doing that over email attachments was never going to scale.

The problem demanded a system that could track changes instead of replacing them. Something that could show differences instead of hiding them. Something that let multiple people work without stepping on each other’s toes, and still allowed everyone to rewind time when things went wrong.

That problem is why version control exists.

Query: "What's our Q3 revenue performance compared to last year?"
Basic RAG Response: "I found some financial information,

Sound familiar? If your RAG system is struggling with complex queries, poor retrieval accuracy, or production scalability issues, you're ready for the next evolution: Advanced RAG.

Modern RAG isn't just about throwing documents into a vector database and hoping for the best. It's a sophisticated orchestration of intelligent query processing, multi-stage retrieval, contextual understanding, and self-correcting generation pipelines. The companies winning with RAG today—from Microsoft's Copilot to Anthropic's Claude—have moved far beyond basic implementations to embrace advanced patterns that can handle the complexity of real-world applications.

This guide dives deep into the cutting-edge techniques transforming RAG from a promising prototype into a production-ready powerhouse.

The Advanced RAG Architecture

Advanced RAG systems operate on three fundamental principles that set them apart from basic implementations:

1. Intelligent Query Processing: Transform user queries into optimal retrieval requests

2. Multi-Modal Retrieval: Combine multiple search strategies and data sources

3. Self-Reflective Generation: Continuously evaluate and improve output quality

Unlike basic RAG's linear "retrieve-then-generate" approach, advanced systems use feedback loops, multi-stage processing, and adaptive decision-making to deliver superior results.

Query Intelligence: Making Every Search Smarter

The foundation of advanced RAG lies in sophisticated query processing. Raw user queries are rarely optimal for retrieval—they're often vague, complex, or use terminology that doesn't match your knowledge base.

Query Rewriting and Expansion

Query rewriting transforms user questions into retrieval-optimized versions. Instead of searching for "AI trends," an advanced system might rewrite this as multiple focused queries: "artificial intelligence trends 2024," "machine learning adoption patterns," and "AI market growth statistics."

Implementation approaches include:

• Zero-shot rewriting: Use LLMs to rephrase queries without examples

• Few-shot rewriting: Provide examples to guide the rewriting process

• Trainable rewriters: Fine-tune specialized models for domain-specific query optimization

Sub-Query Decomposition

Complex questions often contain multiple information needs that require separate retrieval strategies. Advanced RAG systems decompose these into focused sub-queries:

Original: "How did our marketing spend impact customer acquisition in Q3, and what should we budget for Q4?"

Sub-queries:

1. "Marketing spend breakdown Q3 2024"

2. "Customer acquisition metrics Q3 2024"

3. "Marketing ROI analysis Q3 2024"

4. "Q4 marketing budget recommendations"

Each sub-query retrieves targeted context, and the final response synthesizes information from all sources.

Step-Back Prompting

For questions requiring deep domain knowledge, step-back prompting generates higher-level conceptual queries alongside the original question. This retrieves both specific details and broader context, enabling more comprehensive responses.

HyDE: The Game-Changing Retrieval Strategy

Hypothetical Document Embeddings (HyDE) represents one of the most innovative advances in retrieval technology. Instead of directly searching with user queries, HyDE generates hypothetical answers first, then uses these synthetic documents for retrieval.

The HyDE Process:

1. Generate: LLM creates a hypothetical document answering the query

2. Embed: Convert the synthetic document into vector embeddings

3. Search: Find real documents similar to the hypothetical one

4. Filter: Let the encoder remove hallucinated details while preserving relevant patterns

This approach works because answers are more semantically similar to other answers than questions are to answers. A hypothetical response to "How do neural networks learn?" will match actual explanations better than the raw question would.

When to use HyDE:

• Complex technical queries requiring detailed explanations

• Domain-specific questions where terminology matters

• Cases where precision is more important than speed

Hybrid Search: The Best of Both Worlds

Pure vector search excels at semantic understanding but struggles with exact matches, proper nouns, and specific terminology. Keyword search (BM25) handles precise matching but misses semantic relationships.

Hybrid search combines both approaches:

textHybrid Score = (1-α) × BM25_Score + α × Vector_Score

Where α controls the balance between keyword and semantic matching. Advanced systems dynamically adjust this parameter based on query characteristics.

Production implementations typically use:

• Ensemble retrievers: Combine multiple retrieval strategies with weighted scoring

• Rank fusion: Merge results from different search methods using algorithms like Reciprocal Rank Fusion

• Query-adaptive weighting: Adjust the balance based on query type and domain

Contextual Embeddings: Understanding Documents in Context

Traditional embedding approaches encode documents in isolation, losing crucial contextual information. Contextual embeddings address this by incorporating surrounding document context into chunk representations.

Anthropic's Contextual Retrieval approach prepends relevant context to each chunk before embedding:

Original chunk: "The new policy increases efficiency by 15%"
Contextualized chunk: "Q3 2024 Customer Service Policy Update: The new policy increases efficiency by 15%"

This simple technique reduces retrieval failures by 49% and, when combined with reranking, by 67%.

Advanced contextual techniques include:

• Bidirectional context: Incorporate information from preceding and following sections

• Hierarchical context: Include document structure and section headings

• Cross-document context: Link related documents and maintain relationships

GraphRAG: The Knowledge Graph Revolution

While vector databases excel at similarity search, they struggle with complex relationships and multi-hop reasoning. GraphRAG addresses this by structuring knowledge as interconnected graphs rather than isolated chunks.

The GraphRAG Process:

1. Entity Extraction: Identify entities, relationships, and claims from documents

2. Graph Construction: Build a knowledge graph with hierarchical community structure

3. Community Summarization: Generate summaries at multiple levels of granularity

4. Intelligent Querying: Use graph structure for enhanced retrieval and reasoning

GraphRAG Query Modes:

• Global Search: Leverage community summaries for holistic questions about the entire corpus

• Local Search: Fan out from specific entities to their neighbors and relationships

• DRIFT Search: Combine entity-focused search with community context

Microsoft's GraphRAG has shown remarkable improvements in handling complex, multi-entity queries that require reasoning across relationships.

Corrective RAG: Self-Healing Systems

Even with advanced retrieval, systems can return irrelevant or low-quality documents. Corrective RAG (CRAG) introduces self-reflection mechanisms that evaluate retrieval quality and take corrective action when needed.

The CRAG Framework:

1. Relevance Grading: LLM evaluates each retrieved document's relevance to the query

2. Decision Making:

   • Correct documents: Proceed to generation with knowledge refinement

   • Ambiguous/incorrect documents: Trigger web search and query rewriting

3. Knowledge Refinement: Partition documents into "knowledge strips" and filter irrelevant sections

This creates a feedback loop that continuously improves retrieval quality and reduces hallucinations.

LLM-as-a-Judge: Intelligent Evaluation at Scale

Traditional RAG evaluation relies on expensive human assessment or simple metrics that miss nuanced quality issues. LLM-as-a-Judge enables scalable, sophisticated evaluation using specialized judge models.

Evaluation Approaches:

• Single-output scoring: Judge individual responses against defined criteria

• Reference-based evaluation: Compare outputs to gold-standard answers

• Pairwise comparison: Choose the better response between two candidates

Key Metrics for RAG Systems:

• Context Precision: Relevance of retrieved chunks

• Context Recall: Completeness of relevant information retrieval

• Faithfulness: Alignment between generated answers and retrieved context

• Answer Relevancy: How well responses address the original query

Ranking and Reranking Strategies

Initial retrieval often returns relevant documents mixed with noise. Reranking applies more sophisticated models to reorder results based on query-document relevance.

Reranking Approaches:

• Cross-encoder models: Deep interaction between query and document pairs

• LLM-based reranking: Use instruction-following models to assess relevance

• Multi-stage reranking: Combine multiple reranking signals

Production considerations:

• Latency vs. accuracy trade-offs: Cross-encoders are more accurate but slower

• Caching strategies: Store reranking results for frequent queries

• Batch processing: Rerank multiple documents simultaneously for efficiency

Caching: The Performance Multiplier

Production RAG systems must handle thousands of queries per second with sub-second latency. Intelligent caching dramatically improves performance while reducing computational costs.

Caching Strategies:

Query-Based Caching

Store exact retrieval results for specific queries. Perfect for FAQ-style applications with repeated questions.

Semantic Caching

Cache embeddings of previous queries and reuse results for semantically similar new queries. More flexible but requires careful similarity threshold tuning.

Hybrid Caching

Combine exact matching with semantic similarity, using exact matches when available and falling back to semantic similarity.

Production Caching Architecture:

• Redis/Memcached: For high-performance, distributed caching

• TTL policies: Automatic expiration to maintain data freshness

• Cache warming: Precompute results for anticipated queries

• Invalidation strategies: Update caches when underlying data changes

Speed vs. Accuracy: The Production Balancing Act

Real-world RAG systems must balance response quality with performance requirements. Different use cases demand different trade-offs:

High-Speed Scenarios (Customer chat, real-time assistance):

• Smaller, faster embedding models

• Limited reranking

• Aggressive caching

• Simplified query processing

High-Accuracy Scenarios (Research, legal analysis, medical diagnosis):

• Larger, more capable models

• Multi-stage retrieval and reranking

• Extensive query processing

• Quality-focused evaluation

Dynamic Scaling Approaches:

• Query complexity detection: Route simple queries to fast paths, complex ones to comprehensive processing

• User-adaptive systems: Learn individual user patterns and optimize accordingly

• Load-based routing: Switch processing intensity based on system load

Production-Ready Pipeline Architecture

Building RAG systems that work in production requires careful orchestration of all these components. Here's the architecture that powers successful deployments:

Data Ingestion Pipeline

• Multi-format processing: Handle PDFs, web pages, databases, APIs

• Incremental updates: Process new content without full reindexing

• Quality filtering: Remove low-quality or duplicate content

• Metadata enrichment: Add contextual information during ingestion

Retrieval Pipeline

• Query preprocessing: Rewriting, expansion, decomposition

• Multi-stage retrieval: Initial retrieval → reranking → final selection

• Result fusion: Combine outputs from multiple retrieval strategies

• Quality gates: Filter low-quality results before generation

Generation Pipeline

• Context optimization: Arrange retrieved content for maximum relevance

• Response synthesis: Generate coherent answers from multiple sources

• Citation tracking: Maintain links between generated content and sources

• Quality validation: Check responses before returning to users

Monitoring and Observability

• End-to-end tracing: Track queries through the entire pipeline

• Performance metrics: Latency, throughput, accuracy monitoring

• Quality metrics: Automated evaluation using LLM judges

• User feedback loops: Collect and analyze user satisfaction data

The Future of Advanced RAG

As RAG systems mature, several trends are shaping the next generation of capabilities:

Agentic RAG: Systems that can plan multi-step retrieval strategies, use tools, and iterate on results. These systems don't just retrieve and generate—they reason about what information they need and how to get it.

Multimodal Integration: Extending RAG beyond text to include images, audio, video, and structured data. The future of RAG is truly multimodal.

Real-time Learning: Systems that continuously improve from user interactions, updating their knowledge and retrieval strategies based on feedback.

Federated RAG: Architectures that can securely query across multiple organizations' data sources while maintaining privacy and compliance.

Getting Started with Advanced RAG

Ready to upgrade your RAG system? Here's your implementation roadmap:

1. Start with Evaluation: Implement comprehensive metrics before optimizing anything

2. Add Query Intelligence: Begin with simple query rewriting and expand to sub-queries

3. Implement Hybrid Search: Combine vector and keyword search for immediate improvements

4. Deploy Caching: Add semantic caching for performance gains

5. Introduce Self-Reflection: Implement basic relevance grading and corrective mechanisms

6. Scale Gradually: Add complexity incrementally while monitoring performance impact

The Advanced RAG Advantage

The gap between basic and advanced RAG implementations is vast. While naive systems struggle with complex queries and production demands, advanced RAG delivers accurate, contextual, and scalable AI-powered applications.

Companies implementing these advanced techniques report 50-80% improvements in retrieval accuracy, 2-5x faster response times through intelligent caching, and significantly reduced hallucinations through self-corrective mechanisms.

The future belongs to organizations that embrace this complexity and build RAG systems designed for the demands of real-world applications. The techniques covered in this guide—from HyDE and GraphRAG to contextual embeddings and self-reflection—represent the current state of the art, but the field continues evolving rapidly.

Your users expect accurate, intelligent responses. Basic RAG won't cut it anymore. The question isn't whether to adopt advanced RAG techniques—it's how quickly you can implement them to stay competitive in an AI-driven world.

Ready to build production-ready RAG systems? Start with comprehensive evaluation and query intelligence—the foundation of every successful advanced RAG implementation

Yet, despite its theoretical elegance, many RAG systems in production still fail spectacularly. They return irrelevant documents, miss critical information, or worse—confidently generate incorrect answers while citing legitimate sources.

If you've built or worked with RAG systems, you've likely encountered these frustrations. A seemingly perfect setup suddenly starts producing garbage results after a database update. A query that worked yesterday fails today. The system retrieves relevant documents but somehow generates completely wrong answers.

The reality is that RAG systems are complex beasts with multiple points of failure, each capable of undermining the entire pipeline. Understanding where and why these failures occur is crucial for building robust, production-ready systems.

The Anatomy of RAG Failure

RAG failures typically stem from three primary sources: data quality issues, retrieval problems, and generation deficiencies. Each category presents unique challenges that can cascade through your entire system.

1. Poor Recall: When Your System Can't Find What It Needs

Poor recall is perhaps the most fundamental RAG failure. It occurs when your retrieval system fails to find relevant documents that actually exist in your knowledge base. This leads to incomplete context, forcing the language model to either admit ignorance or hallucinate answers.

Common causes include:

• Missing Content: The most basic issue—your knowledge base simply doesn't contain the information needed to answer the query

• Weak Ranking: The relevant documents exist but don't rank highly enough to be retrieved

• Semantic Mismatches: Your query and the relevant documents use different terminology or phrasing, causing the vector search to miss important connections

Quick Mitigation:

• Implement hybrid search combining dense vector search with traditional keyword-based retrieval (BM25)

• Use query expansion techniques to include synonyms and related terms

• Regularly audit your knowledge base coverage by analyzing queries that result in "I don't know" responses

2. Bad Chunking: The Silent Performance Killer

Chunking—breaking large documents into smaller, manageable pieces—seems straightforward but is deceptively complex. Poor chunking strategies can destroy context, break semantic meaning, and create fundamental retrieval problems.

The chunking dilemma:

• Chunks too large: Create coarse representations that obscure important details and may encompass multiple topics, diluting relevance

• Chunks too small: Lose important context and may not contain enough information to be meaningful

• Naive splitting: Character-based splitting can break mid-sentence or separate related concepts

Smart Chunking Strategies:

• Semantic chunking: Understand meaning rather than just splitting text mechanically

• Structure-aware chunking: Respect document organization like headings and sections

• Overlap strategies: Maintain context windows across chunk boundaries

• Domain-specific boundaries: Use "by title" or "by similarity" strategies to preserve topic coherence

Rule of thumb: Start with approximately 250 tokens (1000 characters) per chunk, but always experiment with your specific data.

3. Query Drift: When Questions Don't Match Answers

Query drift occurs when the user's question doesn't align well with how information is stored or indexed in your knowledge base. This mismatch leads to poor retrieval quality, even when relevant information exists.

Manifestations of query drift:

• Vague queries: "AI trends" when the user actually wants "AI trends in medical imaging"

• Complex multi-part questions: Single queries that actually require multiple separate searches

• Domain-specific terminology gaps: User language that doesn't match your indexed content

Query Rewriting Solutions:

• Query expansion: Add synonyms and related terms to broaden search scope

• Query simplification: Break complex queries into focused, manageable parts

• Contextual enrichment: Add relevant context from previous interactions or user profiles

• Multi-query generation: Create multiple variations of the same question to increase retrieval chances

4. Outdated Indexes: The Staleness Problem

One of the most insidious RAG failures is the gradual degradation of performance due to outdated indexes and knowledge bases. As your data evolves but your embeddings remain static, the system becomes increasingly disconnected from reality.

The staleness cascade:

• Information gaps: New data isn't indexed, creating blind spots in retrieval

• Embedding drift: Vector representations become misaligned as content evolves

• Inconsistent retrieval: Updates to vector databases can cause ranking instabilities

Mitigation Strategies:

• Automated update pipelines: Set up systems to detect and process new content regularly

• Drift monitoring: Use clustering techniques or PCA visualization to detect embedding space changes

• Incremental learning: Update embeddings with recent data without full retraining

• Version control: Maintain embedding versions and monitor semantic degradation over time

5. Hallucinations from Weak Context: The False Confidence Problem

Even when RAG systems successfully retrieve relevant documents, they can still generate hallucinations—confident-sounding but incorrect information. This is particularly dangerous because users trust cited sources.

Types of RAG hallucinations:

• Context misinterpretation: The LLM incorrectly processes retrieved information

• Information fusion errors: Incorrectly combining facts from multiple sources

• Citation fabrication: Making up plausible-sounding references

• Confidence misalignment: High confidence in low-quality outputs

Detection and Prevention:

• Multi-metric evaluation: Use frameworks like RAGAS to measure faithfulness, relevance, and context precision

• Semantic similarity checking: Compare generated responses against retrieved context

• Source attribution validation: Verify that citations actually support the claims made

• Hallucination scoring: Implement automated detection using LLM-based evaluators

Building Resilient RAG Systems

Understanding these failure modes is just the beginning. Building production-ready RAG systems requires a systematic approach to prevention, detection, and mitigation.

The Evaluation Framework

You cannot improve what you don't measure. Implement comprehensive evaluation using multiple metrics:

• Context Precision: Measure how relevant your retrieved chunks are

• Context Recall: Ensure you're finding all relevant information

• Faithfulness: Verify generated answers align with retrieved context

• Answer Relevancy: Check if responses actually address the user's question

Monitoring and Observability

Set up monitoring systems to detect failures early:

• Drift detection: Monitor embedding space changes over time

• Performance degradation: Track retrieval quality and response accuracy

• User feedback loops: Collect and analyze user satisfaction data

• Component-level monitoring: Isolate failures to specific pipeline stages

Iterative Improvement

RAG systems require continuous refinement:

• A/B testing: Compare different chunking strategies, embedding models, and retrieval approaches

• User feedback integration: Learn from real-world usage patterns

• Regular audits: Systematically review system performance across different query types

• Proactive updates: Stay ahead of data drift with automated refresh cycles

The Path Forward

RAG isn't broken—it's simply more complex than initial implementations suggested. The key is understanding that RAG systems are not set-and-forget solutions but require ongoing attention, monitoring, and optimization.

Success comes from treating RAG as an engineering discipline rather than a magic solution. This means rigorous testing, comprehensive evaluation, systematic monitoring, and continuous improvement based on real-world performance.

The organizations succeeding with RAG today are those that have invested in robust evaluation frameworks, implemented comprehensive monitoring, and built systems designed for continuous iteration. They understand that the initial deployment is just the beginning—the real work happens in production, where you discover the unique failure modes of your specific use case and systematically address them.

By understanding and preparing for these common failure points, you can build RAG systems that don't just work in demos but deliver reliable, accurate results in the messy reality of production environments. The future belongs to those who can navigate RAG's complexity, not those seeking simple solutions to complex problems.

Ready to build more resilient RAG systems? Start by implementing comprehensive evaluation metrics and monitoring—your future self will thank you when things inevitably break.

As someone who's passionate about AI and its real-world applications, I'm excited to take you on this journey to understand RAG - the technology that's revolutionizing how AI systems access and use information. By the end of this article, you'll not only understand what RAG is but also why it's become the secret sauce behind reliable AI applications.

What is RAG? The AI Detective with a Library Card

Retrieval-Augmented Generation, or RAG for short, is like giving your AI assistant a library card and detective skills rolled into one. Instead of relying solely on what it learned during training (which might be outdated or incomplete), RAG enables language models to actively search for and retrieve relevant information from external sources before generating responses.

Think of it this way: Traditional Large Language Models (LLMs) are like that friend who knows a lot about many topics but sometimes confidently shares "facts" that turn out to be wrong. RAG transforms this friend into a research-savvy individual who always checks their sources before answering your questions.

The Problem RAG Solves

Before RAG came along, LLMs faced several critical limitations:

The Staleness Problem: Models trained on data from 2021 couldn't tell you about events that happened in 2023. Imagine asking about recent stock prices or current news - you'd get outdated information at best.

The Hallucination Dilemma: LLMs sometimes generate convincing-sounding but entirely fabricated information. Studies show that traditional LLMs make mistakes 27% of the time when dealing with specialized questions.

The Knowledge Gap: Your company's internal documents, recent research papers, or proprietary data weren't part of the model's training, leaving huge blind spots in its knowledge.

Why RAG Exists: The Birth of Smarter AI

RAG exists because we needed a way to make AI both knowledgeable and trustworthy. The traditional approach of retraining massive models every time new information becomes available is not just expensive - it's practically impossible for most organizations.

Here's what makes RAG special:

🎯 Access to Real-Time Information

RAG can pull information from databases that are updated daily, hourly, or even in real-time. This means your AI can answer questions about today's stock prices, this week's company announcements, or the latest research findings.

🛡️ Reduced Hallucinations

By grounding responses in actual retrieved documents, RAG reduces AI hallucinations by 60-80% compared to standard LLMs. It's like having a fact-checker built right into your AI system.

💰 Cost-Effective Solution

RAG is significantly more cost-effective than continuously retraining models. It can reduce operational costs by 20% per token compared to traditional LLM approaches.

🔍 Transparency and Traceability

Unlike black-box AI systems, RAG can show you exactly which sources it used to generate an answer, making it possible to verify and trust the information.

The RAG Architecture: Retriever + Generator = Magic

At its heart, RAG consists of two main components working in perfect harmony:

The Retriever: Your AI Research Assistant

The retriever is like having a super-fast librarian who can search through millions of documents in milliseconds. Here's how it works:

1. Query Processing: When you ask a question, the retriever converts your query into a mathematical representation called a vector embedding.

2. Similarity Search: It then searches through a database of pre-computed embeddings to find the most relevant documents or text chunks.

3. Smart Selection: The retriever doesn't just grab random information - it uses sophisticated algorithms to identify the most contextually relevant pieces of information.

The Generator: Your AI Storyteller

Once the retriever has found relevant information, the generator (typically an LLM like GPT-4 or Claude) takes over:

1. Context Integration: The generator receives both your original question and the retrieved information.

2. Intelligent Synthesis: It combines this information to create a coherent, accurate response that directly addresses your query.

3. Source-Grounded Output: The final answer is based on actual retrieved data rather than just the model's training knowledge.

A Simple RAG Example: Let's See It in Action

Let me walk you through a practical example that'll make everything click:

Scenario: You're building a customer support chatbot for a software company.

Step 1: The Question
A customer asks: "What's new in version 3.2 of your product?"

Step 2: Traditional LLM Response
Without RAG, the LLM might say: "I don't have information about version 3.2, as my training data only goes up to early 2023."

Step 3: RAG in Action
With RAG:

1. The retriever searches your company's knowledge base

2. It finds the release notes for version 3.2

3. It retrieves specific information about new features, bug fixes, and improvements

4. The generator crafts a comprehensive response like: "Version 3.2 introduces three major features: enhanced API integration, improved dashboard analytics, and faster data processing. Here are the specific improvements..."

The Result: Your customer gets accurate, up-to-date information that directly answers their question, backed by your actual documentation.

Understanding Indexing: The Foundation of Fast Retrieval

Before RAG can work its magic, there's a crucial behind-the-scenes process called indexing. Think of indexing as creating a super-organized filing system for information.

What is Indexing?

Indexing is the process of organizing and storing information in a way that makes it lightning-fast to search and retrieve. Instead of searching through every single document linearly (which would be painfully slow), indexing creates structures that allow for efficient similarity-based searches.

How Indexing Works

1. Document Processing: Large documents are broken down into smaller, manageable chunks.

2. Vector Conversion: Each chunk is converted into a high-dimensional vector using embedding models.

3. Database Storage: These vectors are stored in specialized vector databases with efficient indexing structures.

4. Search Optimization: When you ask a question, the system can quickly find the most relevant vectors without checking every single one.

Popular Indexing Strategies

Approximate Nearest Neighbors (ANN): Trades a tiny bit of accuracy for massive speed improvements.

Hierarchical Navigable Small World (HNSW): Creates a multi-layer graph structure that balances speed and accuracy perfectly.

Inverted File Index (IVF): Organizes vectors into clusters, making searches in large datasets incredibly efficient.

Vectorization: Turning Words into Math

Here's where things get really interesting. Vectorization is the process of converting text into numerical representations that computers can understand and compare.

Why Vectorization Matters

Imagine trying to determine how similar "king" and "monarch" are. To a computer, these are just different sequences of letters. But through vectorization, both words get converted into numerical vectors that are positioned close to each other in a high-dimensional space, reflecting their semantic similarity.

The Vectorization Process

1. Text Input: "The quick brown fox jumps over the lazy dog"

2. Embedding Model: Advanced neural networks process this text

3. Vector Output: [0.2, -0.8, 1.3, 0.7, ...] (a list of numbers representing meaning)

4. Storage: This vector gets stored in a database for future retrieval

Real-World Impact

This mathematical representation allows RAG systems to understand that:

• "CEO" and "Chief Executive Officer" are similar

• "Python programming" and "coding in Python" are related

• "quarterly earnings" and "Q3 financial results" refer to similar concepts

The Art of Chunking: Breaking It Down Right

Chunking is the process of breaking large documents into smaller, digestible pieces that RAG systems can work with effectively. It's like taking a massive textbook and dividing it into individual pages or sections.

Why Chunking is Crucial

Model Limitations: Embedding models have maximum input sizes (typically around 8,000 tokens or about 6,000 words). A single document might be much larger than this limit.

Search Precision: Smaller chunks allow for more precise retrieval. Instead of returning an entire 50-page document when someone asks about a specific feature, RAG can return just the relevant paragraph.

Context Preservation: Well-designed chunks maintain enough context to be meaningful on their own.

Chunking Strategies

Fixed-Size Chunking: Divide text into equal-sized pieces (e.g., 500 words each).

• Pros: Simple to implement, consistent sizes

• Cons: Might break sentences or paragraphs awkwardly

Content-Aware Chunking: Split based on natural boundaries like paragraphs or sections.

• Pros: Preserves semantic coherence

• Cons: Variable chunk sizes

Semantic Chunking: Use AI to determine where topics change and split accordingly.

• Pros: Each chunk focuses on a single topic

• Cons: More complex to implement

Chunking Best Practices

Size Matters: Typical chunk sizes range from 200-600 tokens with good results.

Think Context: Each chunk should make sense to a human reader without requiring additional context.

Consider Your Use Case: Technical documentation might benefit from section-based chunking, while narrative text might work better with paragraph-based splits.

The Power of Overlapping: Never Lose Context

This is where chunking gets really smart. Overlapping means that consecutive chunks share some common text, typically 10-15% of the content.

Why Overlapping is Essential

Let me illustrate with an example:

Without Overlapping:

• Chunk 1: "Our new product features advanced AI capabilities. It uses machine learning algorithms"

• Chunk 2: "to analyze customer behaviour and provide personalized recommendations."

With Overlapping:

• Chunk 1: "Our new product features advanced AI capabilities. It uses machine learning algorithms to analyze customer behaviour"

• Chunk 2: "machine learning algorithms to analyze customer behaviour and provide personalized recommendations."

The Problem Overlapping Solves

Context Loss: Without overlapping, important information might be split across chunks, making it impossible to retrieve complete answers.

Meaning Preservation: Overlapping ensures that key concepts aren't artificially separated, maintaining semantic coherence.

Better Retrieval: When someone searches for "machine learning customer analysis," both chunks above would be relevant and retrievable with overlapping.

Overlapping Best Practices

Percentage Rule: Start with 10-15% overlap and adjust based on your content type.

Sentence Boundaries: When possible, overlap at complete sentence boundaries rather than mid-sentence.

Content-Specific: Technical documents might need more overlap than simple FAQ content.

Why RAG Wins: The Compelling Advantages

After building and deploying several RAG systems myself, I can tell you that the advantages go beyond just technical improvements:

🔄 Always Current

Unlike traditional models that become outdated the moment they're trained, RAG systems stay current with your latest data. Update your knowledge base, and your AI instantly knows about it.

🎯 Domain Expertise

RAG allows you to create AI systems that are experts in your specific field, whether that's medical research, legal documents, or your company's products.

💡 Transparent Decision Making

When RAG provides an answer, it can cite its sources. This transparency is crucial for business applications where you need to verify information.

🔒 Data Security

Your sensitive information stays in your control. RAG accesses data on-demand rather than incorporating it into model parameters.

⚡ Fast Deployment

Setting up a RAG system is significantly faster than training a custom model from scratch. You can have a working system in days rather than months.

The Road Ahead: RAG's Future

As we stand on the cusp of even more advanced AI capabilities, RAG represents a fundamental shift in how we think about AI knowledge systems. It's not just about having smarter AI - it's about having trustworthy, verifiable, and controllable AI that can adapt to our changing world.

Whether you're building customer support chatbots, internal knowledge systems, or specialized AI assistants, understanding RAG gives you the foundation to create AI applications that are not just impressive, but actually useful and reliable.

The future of AI isn't just about bigger models - it's about smarter architectures that combine the best of human knowledge organization with artificial intelligence capabilities. And RAG is leading that charge.

What aspects of RAG are you most excited to explore in your own projects? Have you encountered the hallucination problem in your AI applications? I'd love to hear your thoughts and experiences in the comments below!

Imagine an AI that doesn’t just answer questions, but thinks, acts, and adapts, planning its own path to achieve real outcomes, making decisions on the fly, and wielding digital tools to get things done. This isn’t science fiction—it’s happening right now with Agentic AI, the next evolution in artificial intelligence that’s captivating both the tech world and mainstream industries. Gartner projects that by 2029, agentic AI will resolve 80% of customer service issues without human intervention, saving businesses billions in costs. But what exactly are agents? How do they work? And what role do tools play in this transformative leap?

Let’s dive deep into the concepts, architectures, and real-world applications that make Agentic AI both fascinating and revolutionary.

What Are Agents in AI—and Why Are They Different?

Agents are autonomous systems designed to operate with limited or zero human oversight, carrying out complex tasks toward predefined goals. Unlike traditional AI that simply reacts to inputs or generates outputs, agents:

• Perceive their environment (through sensors, data, APIs)

• Reason about goals and constraints

• Act by executing external functions, APIs, or physical controls

• Learn and adapt continuously through feedback and memory

• Collaborate with other agents, forming multi-agent teams

In short, an agent isn’t just a passive bot or a chatbot—it’s a decision-maker that can break objectives into steps, adjust plans, and even use external digital tools to complete the task at hand.

How Do Agents Work? - The Architecture of Intelligence

At the core of every agent lies a set of interconnected modules. Here’s how they operate in a continuous loop:

Agents combine all these layers to handle tasks from the simple (“book a meeting”) to the wildly complex (“optimize a supply chain during a global disruption”).

Agent Architectures: Models in Action

• Reactive Agents: Fast yet limited; respond instantly to environment (e.g., thermostat)

• Model-Based Agents: Maintain internal models to predict outcomes (e.g., Roomba, mapping routes)

• Goal-Based Agents: Plan multi-step strategies (e.g., self-driving cars, route planning)

• Utility-Based Agents: Optimize for maximum benefit (e.g., Uber dynamic pricing)

• Learning Agents: Adapt and refine over time (e.g., smart email spam filters)

• Multi-Agent Systems: Many agents cooperate or compete to solve large-scale tasks (e.g., air traffic control)

The Role of Tools—From Function Calling to Real-World APIs

The real magic of agentic AI lies in its capacity for tool use—connecting reasoning models with external functionalities. Think of it as giving the agent “digital hands”.

Tool Use & Function Calling

Agentic AI employs function calling—where an LLM (Large Language Model) identifies when to invoke an external tool, API, or custom function. The process:

1. Recognize Need: The agent parses a user request (e.g., “What’s the weather in Delhi?”).

2. Decide on Action: The LLM determines an API or tool (e.g., get_current_weather).

3. Execute Function: The agent calls the tool/API, retrieves the data.

4. Integrate Results: Combines the fetched info into a final answer.

Tools can be business apps, external APIs, code editors, databases, even other agents—making agents capable of writing reports, booking resources, running analyses, and more.

Multi-Agent Frameworks like LangChain, AutoGen, CrewAI, and LangGraph have emerged, allowing developers to build agents that broker tasks, share memory, and even coordinate using role-based collaborations. This modularity means tools can be constantly upgraded—making agents endlessly extensible.

Memory and Learning: How Agents "Remember"

Unlike static models, agents use vector databases, knowledge graphs, and neural memory modules to store and recall information from previous tasks, user sessions, and even other agents. This enables personalization (“Remember my preferences?”), context retention for long conversations, and lifelong learning. Memory architectures incorporate:

• Short-term (session context)

• Long-term (persistent user data)

• Episodic (tracking sequence of actions/events)

• Semantic (general facts & learned knowledge)

• Procedural (decision rules & task steps)

Frameworks like LangChain use vector stores (Pinecone, ChromaDB) to give agents real semantic recall, not just keyword matching.

How Agents Reason and Act: The ReAct Paradigm

The cutting-edge ReAct agent design blurs the lines between thought and action. Rather than simply returning an answer, a ReAct agent alternates between:

• Thought: Internally reasons about the next step

• Action: Executes a tool/API call or searches knowledge

• Observation: Integrates external results into reasoning

This cycle repeats, just like a human solving a problem step-wise, until the agent is satisfied with the outcome or reaches the final solution.

Here’s an example trace:

textThought: I need to fetch today’s weather for Mumbai.
Action: Call get_current_weather(Mumbai)
Observation: Sunny, 34°C
Thought: Advise the user on what to pack.
Final Answer: It’s sunny in Mumbai at 34°C—don’t forget sunglasses and water!

This loop makes agents flexible, self-correcting, and highly adaptive—able to consult references online, calculate, search, and synthesize.

Real-World Applications: Where Agentic AI Already Rules

Agentic AI isn’t just a buzzword; it’s deeply embedded in the products and services people use daily—even if they don’t notice:

More industries are embracing agentic AI every day, unlocking efficiencies and new user experiences.

Closing Thoughts: The Future of Agentic AI Is Bang in the Middle of Your Daily Life

Agentic AI isn’t just reshaping software—it’s changing the very fabric of how businesses and humans interact with technology. Autonomous, goal-driven, endlessly adaptive, and tool-equipped—agents are the architects of tomorrow’s intelligent workflows.

If you’re a developer, the message is clear: mastering agentic systems, multi-agent frameworks, and tool integration is the fast track to building the next generation of digital experiences.

And for everyone else? You’re already living in an agentic world—ask your virtual assistant what it did to fulfil your last request, and you’ll get a glimpse of how many agents just collaborated to make it happen.

Catchy Headline Variations:

• "Agentic AI Unleashed: Autonomous Agents and the Tools That Make Them Superhuman"

• "From Chatbots to Autonomous Decision-Makers: The Agentic AI Revolution Explained"

• "Inside Agentic AI: How Digital Agents Plan, Act, and Shape the Future"

Keep following the AI space—it’s not just getting smarter; it’s getting agentic.