Training Your Chatbot Using URLs

Prerequisite: You must be logged in to your Thinkstack account and have already created a chatbot. Select the chatbot for which you want to train and follow these steps:

1. Go to the 'Sources' section in the top navigation bar, and ensure 'URL' is selected in the sidebar.
2. Copy the URL of the website you want to use as the data source for training and paste it as shown in the image below.
   

   Note: You can exclude specific pages that are not needed for training by using glob patterns, or later by unchecking them from the list of URLs once the crawling process is complete.
   

3. Click on 'Start Crawling' to begin the process. This will automatically import all pages from the provided URL. You can view the status of the crawling process as shown below.
4. Once the crawling is complete, Click on 'Train' to initiate the training process.

   Note: You can include additional domains in the training process by clicking on 'Add Domain & Train.' This allows you to expand the data sources used for training the chatbot, ensuring it has comprehensive coverage from multiple websites..

5. Review the list of URLs and uncheck any that you don’t want to include in the training.
6. When ready, click 'Train now' to start the training. The time required will depend on the number of pages. The domain will be marked as 'trained' once the training is complete.
7. Once the training is complete, test your chatbot by asking it questions to see how well it responds. That's a wrap!🎉, You have successfully trained your chatbot using a URL.

📺Watch the video to learn how to train your AI chatbot with URL.

Updating Your Knowledge Base

Your website is constantly evolving, with new information or data being added. To keep your chatbot updated with these changes, there are two ways to update its knowledge base:

1. Adding new pages manually
   
2. Comparing changes using a sitemap
   

Let’s understand when and how to use these methods.

If you are adding a few new pages, you can manually update the chatbot’s knowledge base by adding their URLs.

Add a New Page Manually

1. Go to Sources > Select your domain > Click Update.
2. Choose Add New Page.
3. Enter the URL of the new page and click Proceed.

That's it! The new page will be crawled and added to the chatbot’s knowledge base.

However, if there are significant changes to your website, such as multiple pages being added, removed, or updated, ensure these changes are reflected in your sitemap. You can then update the chatbot's knowledge base using the following steps:

Compare Changes Using a Sitemap

1. Go to Sources > Select your domain > Click Update.
2. Select Compare Changes.
3. Enter your sitemap URL (e.g. : https://yourdomainname.com/sitemap.xml) and click Proceed.

   note

   When you add a sitemap, the system compares its URLs with the ones already present in your chatbot’s knowledge base. This process detects new, updated, or removed content, allowing you to sync either of these changes with a single click.

4. The system will compare the existing knowledge base with the sitemap and display the changes:
   

   note

   🔹 New Pages: Pages recently added to your site.
   🔹 Updated Pages: Pages with recent modifications.
   🔹 Removed Pages: Pages no longer in the sitemap.

5. Select the pages you want to update and click Update All to sync the changes one click. New and updated pages will be crawled, while removed pages will be deleted from the knowledge base.

That's it! Your chatbot’s knowledge base is now successfully updated.

After updating, ask the chatbot questions related to the newly added information to verify that it responds correctly.