Robots file

The robots file contains information for search engine robots about which pages of your website may not be indexed and listed in search engines. The robots file can be generated automatically by the shop system. The file is automatically created for the first time during the installation routine.

If you move your online shop to a different directory on the web server, or make your shop accessible under a different internet address, the robots file must be updated. To do this, go to Settings / Search engine optimization / Settings for search engine bots (robots.txt).

When generating the robots file, the shop-side contents (Content > Content Manager, see chapter Content Manager) are checked to see whether the setting Entry in robots.txt (disallow) is activated. If this is the case, the link to the corresponding content is entered in the robots file with a so-called Disallow. The same applies to the entries in the Static pages tab (see chapter Static pages). The Disallow entry instructs search engines not to include the page in question in the index.

Attention

Pages with legal information, such as the right of withdrawal, should always be included in the robots file with a Disallow. If a search engine indexes such a page, the situation may arise that an outdated version of the legal text is kept by the search engine. Under certain circumstances, this can lead to a formal warning (Abmahnung).

Two buttons are available for generating a robots file:

Generate and save

Clicking this button generates the robots file and updates it directly on the FTP server.

Generate and download

This button generates a new robots file and offers it for download. It can be adjusted if necessary before it is manually uploaded to the FTP server. To do this, connect to your FTP server via an FTP program and upload the file to the directory that is displayed to the left of the button.