IndexNow for better SEO

For SEO(Search Engine Optimization), we usually use sitemap.xml. Microsoft also supports sitemap.xml, but it looks like that they want to have the different kind of standard which is named as IndexNow.

IndexNow is an initiative that was launched by Microsoft in collaboration with Yandex. It allows websites to instantly notify search engines about content changes (such as updates, new pages, or deletions) using a standardized API.

This reduces the time it takes for search engines to crawl and reflect website changes in their indexes. While it was initially spearheaded by Microsoft Bing and Yandex, other search engines, such as Seznam, have also joined the effort, though Google has not yet fully adopted it.

So, Microsoft and Yandex can be considered the primary owners of the IndexNow protocol.

Steps to Set Up IndexNow on CentOS 7 using curl:

1. Generate an API Key: To interact with IndexNow, you first need to generate an API key. This is typically a unique key for your domain.

   You can create a 32-character API key using any method, such as openssl:

   openssl rand -hex 16
   

   Save this key in a file on your website at https://yourdomain.com/indexnow.txt.

2. Create a JSON Payload: For every URL you want to submit to IndexNow, create a JSON payload that includes the URL(s) and your API key.

   {
     "host": "yourdomain.com",
     "key": "YOUR_API_KEY",
     "keyLocation": "https://yourdomain.com/indexnow.txt",
     "urlList": [
       "https://yourdomain.com/new-page",
       "https://yourdomain.com/updated-page"
     ]
   }
   

3. Submit URLs to IndexNow using curl:

   Use curl to send the request to the IndexNow endpoint. Replace "YOUR_API_KEY" with the key you generated and "yourdomain.com" with your actual domain.

   Example curl command:

   curl -X POST \
   -H "Content-Type: application/json" \
   -d '{
     "host": "yourdomain.com",
     "key": "YOUR_API_KEY",
     "keyLocation": "https://yourdomain.com/indexnow.txt",
     "urlList": [
       "https://yourdomain.com/new-page",
       "https://yourdomain.com/updated-page"
     ]
   }' https://www.bing.com/indexnow
   

4. Automate URL Submission (Optional): If you'd like to automate URL submissions, you can set up a cron job on CentOS 7 to trigger the curl request whenever content changes.

   Example cron job (submit every 5 minutes):

   */5 * * * * curl -X POST -H "Content-Type: application/json" -d '{
     "host": "yourdomain.com",
     "key": "YOUR_API_KEY",
     "keyLocation": "https://yourdomain.com/indexnow.txt",
     "urlList": [
       "https://yourdomain.com/new-page",
       "https://yourdomain.com/updated-page"
     ]
   }' https://www.bing.com/indexnow
   

Key Notes:

• Ensure the indexnow.txt file is accessible at the URL specified.
• Keep your API key secure, but publicly shareable in the indexnow.txt file.
• Check the documentation for other search engines using IndexNow (e.g., Yandex, Bing).

This approach will allow your CentOS 7 system to interact with the IndexNow protocol using curl.

Key Takeaways: IndexNow for Faster SEO

• IndexNow is a new standard for notifying search engines about website changes. This allows search engines to update their indexes more quickly, improving your website's visibility.
• IndexNow is a collaborative effort, currently led by Microsoft Bing and Yandex. Other search engines, such as Seznam, are also participating, but Google has not yet fully adopted it.
• Implementing IndexNow requires a few simple steps:
  • Generate a unique API key and store it in a publicly accessible file on your website.
  • Create a JSON payload containing your URLs and API key.
  • Submit the payload to the IndexNow endpoint using curl.
• For optimal results, consider automating the submission process. This can be done by setting up a cron job that runs periodically and submits new or updated URLs.
• Keep your API key secure, but make it accessible via the specified file location. This allows IndexNow to authenticate your requests.
• Refer to the documentation of participating search engines for specific instructions and requirements.