The Add website page is where you create a new website record in your local Siteimp workspace. It asks for the minimum information Siteimp needs before it can prepare a site for scanning: a clear name and a start URL.

This page is also where Siteimp begins setting expectations for the scan. Before the website is saved, the page can preview robots.txt, hold optional crawl setting overrides, and explain ownership verification. After the website is saved, ownership verification becomes active and the site is ready to move into its dashboard.

The most important thing to understand is that adding a website does not run a scan by itself. It creates the local website record first. From there, you can verify ownership, review scan behavior, and run the first scan when you are ready.

What this page is for

The Add website page exists to answer one practical question:

What site should Siteimp know about before it starts collecting evidence?

A website record gives Siteimp a stable local home for:

  • the website name
  • the primary start URL
  • the registrable domain
  • crawl settings for this site
  • ownership status
  • future scan history
  • future monitoring setup

This keeps setup separate from scanning. Siteimp can save the website first, then use the website dashboard and scan results pages for deeper evidence.

What you can do here

Add a website name

The Name field is the label Siteimp uses in your dashboard and scan history. It does not have to match the domain exactly. Use a name that will make sense when you return to the app later.

Good examples:

  • Formimp
  • StorePhotos.ca
  • Client marketing site
  • Main company website

The name is for your local workspace. It helps you recognize the site quickly, especially once you have more than one website in Siteimp.

Enter the start URL

The Start URL is the primary URL Siteimp should begin from. Use the full address, including https:// or http://.

For example:

https://example.com/

The start URL matters because Siteimp uses it to understand the website’s starting point. It also gives the page enough information to preview robots.txt before the first scan.

Preview robots.txt

The Robots.txt preview card appears on the right side of the page. Before a valid start URL is entered, it waits for a start URL. Once the start URL is valid, Siteimp checks for a robots.txt file and summarizes what it finds.

The preview can show information such as:

  • whether robots.txt was found
  • the fetched URL
  • the final location, if redirects were involved
  • the matched user agent
  • crawl delay
  • allow and disallow rule counts
  • sitemap locations
  • a raw robots.txt preview

This preview is only a guide. Siteimp checks robots.txt again right before an actual scan starts, so the final scan uses the current policy at scan time.

Set optional crawl settings

The Crawl settings card lets you set draft website-specific crawl behavior before the website exists.

These values are optional. Leaving them blank means Siteimp can use the broader app defaults.

The available draft settings may include:

  • crawl delay override
  • crawl jitter override
  • max concurrency override

These settings are about politeness and scan behavior. A delay controls the minimum wait between requests. Jitter adds a small random extra delay. Concurrency controls how many crawl requests can run at the same time when that option is shown.

On this setup page, these are draft values. They are applied right after the website is created.

Review ownership status

Before the website is saved, the Ownership card explains that ownership verification is not active yet. This is expected. Siteimp needs a saved website record before it can generate a verification challenge.

Ownership verification matters because it unlocks owner override and advanced scanning options for a site you own or control.

Before save, the card explains the available methods:

  • Meta Tag, usually the fastest option
  • DNS TXT, a strong long-term option

After save, the card becomes live and can generate verification challenges.

How to use this page

A good first setup flow is:

  1. Enter a clear website name.
  2. Enter the full start URL.
  3. Review the robots.txt preview when it appears.
  4. Leave crawl settings blank unless you already know you want custom behavior.
  5. Save the website.
  6. Verify ownership now, or continue to the website dashboard and verify later.
  7. Run the first scan from the website dashboard when you are ready.

For most websites, the safest path is to fill in the name and start URL first, review the side cards, then save without changing advanced settings.

Required fields

Name

The name is required because Siteimp needs a readable label for the website in your local dashboard and scan history.

This field is not sent to a cloud service by Siteimp. It is part of your local workspace.

Start URL

The start URL is required because Siteimp needs a primary address for the website.

The URL must be a complete address. Use:

https://example.com/

instead of:

example.com

A full URL helps Siteimp avoid guessing which scheme, host, or path you meant.

What happens when you save

When you choose Save website, Siteimp validates the form first.

If the fields are valid, Siteimp:

  1. creates the website record
  2. applies any draft crawl settings
  3. reloads the saved website details
  4. switches the page into the Website created state

The created state shows a website summary with:

  • name
  • start URL
  • registrable domain
  • website key

At this point, the website is part of your Websites dashboard and ready for scan setup.

Website created state

After the website is saved, the page changes from a form into a confirmation view.

From there, you can:

  • choose Add another to reset the form and add another website
  • choose Continue to dashboard to open the new website dashboard
  • use the now-active Ownership card to verify the website

The ownership card is open by default after creation because verification is often the next useful setup step.

Ownership verification

Ownership verification proves that you control the website.

Siteimp supports two verification methods:

Meta Tag

Meta tag verification is usually the fastest option. Siteimp generates a meta tag that you add to your homepage HTML. After adding the tag, choose Verify now.

The tag must be present in the server-delivered HTML. If the tag is added later by client-side JavaScript, Siteimp may not be able to verify it.

DNS TXT

DNS TXT verification is a strong long-term option. Siteimp generates a TXT record name and value that you add to your DNS settings. After the record is in place, choose Verify now.

DNS changes can take time to propagate. If verification does not pass right away, wait a little longer and try again.

Owner override and scan policy

Until ownership is verified, advanced scanning is unavailable and the scan policy remains unavailable for owner override.

After ownership is verified, Siteimp can let you choose how future scans behave:

  • Respect robots.txt keeps scans in strict mode.
  • Use owner override allows advanced scanning for the verified website.

For most sites, respecting robots.txt is the safest default. Owner override is for websites you own or control where you intentionally want Siteimp to scan with verified-owner permissions.

Robots.txt preview

Waiting for start URL

Before a start URL is entered, the preview waits. This is normal. Siteimp cannot know where to look for robots.txt until it has a valid website address.

Checking robots.txt

After you enter a valid start URL, Siteimp waits briefly and then checks for robots.txt. This small delay helps avoid checking while you are still typing.

Robots.txt available

If Siteimp finds and reads robots.txt, the card can show a summary and a details view. Use Show details to inspect the fetched URL, active rules, sitemaps, and raw text preview.

No robots.txt found

If no robots.txt file is found, Siteimp reports that in the preview. This does not necessarily mean the website cannot be scanned. It means the site did not publish a robots.txt file at the expected location.

Robots.txt needs review

If robots.txt was fetched but some rules could not be interpreted cleanly, Siteimp flags that the file needs review. This is a signal to inspect the rules before relying on the preview.

Robots preview unavailable

If the preview is unavailable, Siteimp could not check robots.txt at that moment. This may happen because of a network problem, a server response issue, a temporary failure, or a URL that cannot be reached.

The preview is helpful, but it is not the final scan decision. Siteimp checks robots.txt again before scanning.

Crawl settings

The crawl settings card is for website-specific politeness rules.

Crawl delay override

This is the minimum wait between requests before jitter is applied. A larger delay makes the crawl gentler but slower.

Crawl jitter override

This adds random extra delay per request. Jitter can help avoid sending requests in an overly mechanical pattern.

Max concurrency override

When shown, this controls the maximum number of simultaneous crawl requests. Using 1 is the safest polite default.

Leaving fields blank

Blank fields mean “do not set a website-specific override.” In that case, Siteimp can use its app-level defaults instead.

Validation messages

Website name needed

This message appears when you try to save without entering a name.

Add a short name for the website, then save again.

Start URL needed

This message appears when you try to save without entering a start URL.

Enter the primary URL Siteimp should start from. Include the full address.

Start URL needs a full address

This message appears when the start URL is not a complete URL.

Use a full address with http:// or https:// at the front.

For example:

https://example.com/

Troubleshooting

Website was not saved

If Siteimp says Website was not saved, the app could not create the website record.

Try these steps:

  1. Confirm the name is filled in.
  2. Confirm the start URL is a complete URL.
  3. Check whether the website already exists in your workspace.
  4. Try saving again.
  5. If the error continues, contact support and include what you were trying to add.

Crawl settings not saved

If crawl settings fail to save, the website-specific override may not have been applied.

Check that numeric fields contain whole numbers. Delay and jitter must be 0 or higher. Concurrency, when available, must be 1 or higher.

You can still save the website without custom crawl settings and adjust them later from the website dashboard or settings area.

Ownership not verified yet

This means Siteimp checked the ownership challenge but did not find what it needed yet.

For meta tag verification:

  • confirm the tag was added to the homepage
  • confirm it is present in the server-delivered HTML
  • try verifying again

For DNS TXT verification:

  • confirm the TXT record name and value match the generated challenge
  • wait for DNS propagation
  • try verifying again

Ownership check failed

This means Siteimp could not complete the ownership check.

The cause may be a network problem, DNS issue, server issue, or unexpected response. Review the error shown in the app, then try again. If it keeps failing, contact support with the website name, verification method, and what happened.

Where to go next

After adding a website, the next destination is usually the website dashboard.

From there, you can:

  • review the saved website
  • verify ownership if you did not do it here
  • review scan setup
  • run the first scan
  • use Scan Results as the evidence hub

The Add website page is the setup step. The website dashboard is where the website becomes operational. The scan results pages are where Siteimp starts showing the deeper evidence.