Quarterly Incremental WHOIS Data Manual Whois API LLC https://www.whoisxmlapi.com Copyright ©2010-2022 This database is licensed to you or your organization only, you may not resell or relicense the data without explicit written permission from Whois API LLC. Any violation will be prosecuted to the fullest extent of the law. Please go to https://www.whoisxmlapi.com/support/WhoisAPIDatabaseSLA.pdf to view the complete license agreement. About this document This document describes quarterly gTLD and ccTLD incremental WHOIS data which are available for download in advance, before a new quarterly release. File URLs: * https://www.domainwhoisdatabase.com/docs/Quarterly_Incremental_WHOIS_Data_Manual File version 1.5 Approved on 2022-04-11. A full list of available WhoisXML API data feed manuals is available at https://www.domainwhoisdatabase.com/docs Contents * 1 About incremental data * 2 File formats, release versions * 3 Data types and locations * 3.1 The “all_files” subdirectory * 3.2 The “status” subdirectory * 3.3 Update notification RSS 1 About incremental data Incremental data are provided to enable a preliminary access to the contents of a future quarterly database release before the release is actually issued. Incremental data are available for both the gTLD and ccTLD databases; the present document describes both. The data are generated typically once a week, at the end of the week, but the data generation has no strict schedule. The data contents, i.e. the TLDs for which data are published, and the subset of domains within them is determined by the internal logics of the data collection procedure, which is of stochastic nature. Hence, the sevice is provided as it is, there are no declarations about the schedule and completeness of them. 2 File formats, release versions The data are provided in gzip-compressed csv files. There are three types of csv files: “simple”, “regular”, and “full”. Please consult the “Quarterly gTLD WHOIS Database Reference Manual” at https://www.domainwhoisdatabase.com/docs/Quarterly_GTLD_WHOIS_Database_Reference_Manual and the “Quarterly ccTLD WHOIS Database Reference Manual” at https://www.domainwhoisdatabase.com/docs/Quarterly_CCTLD_WHOIS_Database_Reference_Manual for their detailed description. Quarterly releases are identified by a “release version”, e.g. “v39”, this is also explained in detail in the cited manual. 3 Data types and locations Incremental data are located under the following directory: gTLDs: https://www.domainwhoisdatabase.com/whois_database/incremental_files ccTLDs: https://www.domainwhoisdatabase.com/domain_list_quarterly/incremental_files 3.1 The “all_files” subdirectory Each gzipped csv file contains data for a given top-level domain (TLD). The data are provided here for those top-level domains which are supported by the service of WhoisXML API, Inc. The data for bigger TLDs is split into multiple files, each containing up to 100,000 records and identified by an ordinal number. The files are named as $version_csv_$csvtype_$tld_$n.csv.gz where $version is the release version, $tld is the TLD, and $csvtype is the type of the file, one of “simple”, “regular”, or “full”, while $n is the ordinal number of the file belonging to this TLD. e.g.: v19_csv_full_aero_1.csv.gz. Each file is accompanied with an .md5 and .sha256 checksum file in support of download verification. 3.2 The “status” subdirectory The status subdirectory is to be found in the all_files subdirectory and contains files providing information on the available data files. These are named according to the release version they belong to, e.g. vXX_exported_files, e.g. v39_exported_files belongs to the release version v39. The file without a version number, ’exported_files’ is identical to the most recent one. Older files are removed. The exported_files files are log files containing metadata for all quarterly releases from release version v14 on, till the actual one which is in preparation. The format of the lines of the file is FILENAME BYTES DATE_CREATED TIME_CREATED with the following elements: FILENAME : The name of the file the record refers to, in the format described previously, e.g. v39_csv_simple_app_6.csv.gz is the 6th collection of up to 50,000 csvs for the TLD .app, in the csv format “simple”, in release “v39”. BYTES The size of the file in bytes. DATE_CREATED The date when the creation of the file was accomplished, in the format of MM-DD-YYYY. TIME_CREATED The time in the 24 hour format when the creation of the file was accomplished, in the format of HH:MM:SS. Example of a full line of the file: v39_csv_simple_app_6.csv.gz 6171653 2022-04-05 08:45:41 Filename: v39_csv_simple_app_6.csv.gz File size: 6171653 File created date: 2022-04-05 File created time: 08:45:41 Important: If there is no file creation date and time for a particular .gz file in the exported_files log, the file creation date and time can be read from the file date of the respective .md5 or .sha256. 3.3 Update notification RSS An RSS feed is available in the data directory root which emits notifications whenever a bunch of newly upoloaded data files is available. The RSS URLs are: gTLDs: https://www.domainwhoisdatabase.com/whois_database/incremental_files/rss/update_notifications_rss.xml ccTLDs: https://www.domainwhoisdatabase.com/domain_list_quarterly/incremental_files/rss/update_notifications_rss.xml The RSS feeds can be accessed with any RSS client (e.g QuiteRss) using the same login credentials as for the data themselves. The RSS messages can also be used to trigger automated downloading and processing of the data, see e.g. this technical blog for a similar case study. ---------------------------------------------------------------------- This document was translated from L^AT_EX by H^EV^EA.