This guide will show you how to set up a data backfill request to add historical data to your project with Supermetrics for BigQuery.
Before you begin
- Make sure the initial transfer has been completed successfully for your data source of choice: How to configure a data transfer in BigQuery
- Trial licenses can only backfill up to 14 days. If you try to backfill farther than that, the transfer will fail.
- You can bulk transfer up to 6 months worth of data at one time. If you need more time than that, you will need to do it in separate, 6-month-sized batches.
- Different data sources have different data retention policies, so you may only be able to fetch up to their maximum value. See their API documentation for further details.
Schedule a backfill
1. Open the BigQuery project you have connected to the Supermetrics data source, and go to Data Transfers.
2. Find the transfer in the list you want to use for the backfill request. Click on it to open the Transfer details page.
3. In the top right corner, click on SCHEDULE BACKFILL.
4. Set the Start Date and End Date values for the historical data you want to transfer into the dataset.
- Note that you can only do backfills with a maximum time frame of 6 months at once, so you may need to do the backfill in batches for longer periods. Some data sources may not be able to handle requests for the current date.
5. Click on OK to start the transfer process.
6. For each day in the range, a new transfer request will be added in the RUN HISTORY section. This may take some time to complete, depending on how much data you requested at once. You can monitor the progress there in BigQuery or in Team Management – see How to monitor your data transfers.
Once all the requests are complete, you can see the additional tables for each day in the backfill in the dataset and utilize the data in your queries and reporting.
- Check out the Data Warehouse Table descriptions to see what is available for other sources
- How to monitor your data transfers in Team Management
- Visualizing your BigQuery data in Data Studio
Need help? Contact Supermetrics Support