Datasetpublic

hombit/ztf-bts-xmatch

TDABench ZTF BTS HATS Catalog Collection TODO: Add description. ZTF BTS crossmatched with multiple optical catalogs Access the catalog We recommend the use of the LSDB Python framework to access HATS catalogs. LSDB can be installed via pip install lsdb or conda install conda-forge::lsdb, see more details in the docs. The following code provides a minimal example of opening this catalog: import lsdb # Full sky coverage. catalog =… See the full description on the dataset page: https://huggingface.co/datasets/hombit/ztf-bts-xmatch.

sourceHugging Faceupdated 7mo agoView on Hugging Face

0likes42downloads

Dataset Card

configs:

—configname: default datadir: bts-xmatch/dataset
—configname: bts-xmatch5arcs datadir: bts-xmatch5arcs/dataset tags:
—astronomy ---

TDABench ZTF BTS HATS Catalog Collection

TODO: Add description. ZTF BTS crossmatched with multiple optical catalogs

Access the catalog

We recommend the use of the LSDB Python framework to access HATS catalogs. LSDB can be installed via pip install lsdb or conda install conda-forge::lsdb, see more details in the docs. The following code provides a minimal example of opening this catalog:

python

import lsdb

# Full sky coverage.
catalog = lsdb.open_catalog("hf://datasets/hombit/ztf-bts-xmatch")
# One-degree cone.
catalog = lsdb.open_catalog(
    "hf://datasets/hombit/ztf-bts-xmatch",
    search_filter=lsdb.ConeSearch(ra=135.0, dec=1.0, radius_arcsec=3600.0),
)

Each catalog in this collection is represented as a separate Apache Parquet dataset and can be accessed with a variety of tools, including pandas, pyarrow, dask, Spark, DuckDB.

File structure

This catalog is represented by the following files and directories:

—`collection.properties` — textual metadata file describing the HATS collection of catalogs
—`bts-xmatch` — main HATS catalog directory
—`dataset/` — Apache Parquet dataset directory for the main catalog
—... parquet metadata and data files in sub directories ...
—`hats.properties` — textual metadata file describing the main HATS catalog
—`partition_info.csv` — CSV file with a list of catalog HEALPix tiles (catalog partitions)
—`skymap.fits` — HEALPix skymap FITS file with row-counts per HEALPix tile of fixed order 8
—`bts-xmatch_5arcs/` — default margin catalog to ensure data completeness in cross-matching, the margin threshold is 5.0 arcseconds
—... margin catalog files and directories ...

Catalog metadata

Metadata of the main HATS catalog, excluding margins and indexes:

Number of rows	Number of columns	Number of partitions	Size on disk	HATS Builder
10,311	21	12	5.2 MiB	lsdb v0.8.3.dev10+g8d9bf935a, hats v0.8.3.dev4+gc987c6081

Catalog columns

The main HATS catalog contains the following columns:

Name	`ZTFID`	`IAUID`	`RA`	`Dec`	`peakt`	`peakfilt`	`peakmag`	`peakabs`	`duration`	`rise`	`fade`	`type`	`redshift`	`b`	`A_V`	`legacysurvey.FLUX_G`	`legacysurvey.FLUX_IVAR_G`	`legacysurvey.FLUX_R`	`legacysurvey.FLUX_IVAR_R`	`legacysurvey.FLUX_I`	`legacysurvey.FLUX_IVAR_I`	`legacysurvey.FLUX_Z`	`legacysurvey.FLUX_IVAR_Z`	`legacysurvey.RA`	`legacysurvey.DEC`	`legacysurvey._dist_arcsec`	`desi.Z`	`desi.ZERR`	`desi.TARGET_RA`	`desi.TARGET_DEC`	`desi._dist_arcsec`	`ps1.objID`	`ps1.gMeanKronMag`	`ps1.gMeanKronMagErr`	`ps1.rMeanKronMag`	`ps1.rMeanKronMagErr`	`ps1.iMeanKronMag`	`ps1.iMeanKronMagErr`	`ps1.zMeanKronMag`	`ps1.zMeanKronMagErr`	`ps1.raMean`	`ps1.decMean`	`ps1._dist_arcsec`	`wise.source_id`	`wise.ra`	`wise.dec`	`wise.sigra`	`wise.sigdec`	`wise.w1mpro`	`wise.w2mpro`	`wise.w3mpro`	`wise.w4mpro`	`wise.w1sigmpro`	`wise.w2sigmpro`	`wise.w3sigmpro`	`wise.w4sigmpro`	`wise.w1rchi2`	`wise.w2rchi2`	`wise.pmra`	`wise.pmdec`	`wise.sigpmra`	`wise.sigpmdec`	`wise._dist_arcsec`	`gaia.source_id`	`gaia.ra`	`gaia.ra_error`	`gaia.dec`	`gaia.dec_error`	`gaia.parallax`	`gaia.parallax_error`	`gaia.pm`	`gaia.pmra`	`gaia.pmra_error`	`gaia.pmdec`	`gaia.pmdec_error`	`gaia.phot_g_mean_mag`	`gaia.phot_bp_mean_mag`	`gaia.phot_rp_mean_mag`	`gaia.phot_g_n_obs`	`gaia.phot_bp_n_obs`	`gaia.phot_rp_n_obs`	`gaia.ruwe`	`gaia.phot_bp_rp_excess_factor`	`gaia._dist_arcsec`	`lightcurve_yse.band`	`lightcurve_yse.time`	`lightcurve_yse.flux`	`lightcurve_yse.flux_err`	`_healpix_29`
Data Type	string	string	double	double	double	string	double	string	string	string	string	string	string	double	double	list[float]	list[float]	list[float]	list[float]	list[float]	list[float]	list[float]	list[float]	list[double]	list[double]	list[double]	list[double]	list[double]	list[double]	list[double]	list[double]	list[int64]	list[double]	list[double]	list[double]	list[double]	list[double]	list[double]	list[double]	list[double]	list[double]	list[double]	list[double]	list[string]	list[double]	list[double]	list[double]	list[double]	list[double]	list[double]	list[double]	list[double]	list[double]	list[double]	list[double]	list[double]	list[double]	list[double]	list[double]	list[double]	list[double]	list[double]	list[double]	list[int64]	list[double]	list[float]	list[double]	list[float]	list[double]	list[float]	list[float]	list[double]	list[float]	list[double]	list[float]	list[float]	list[float]	list[float]	list[int16]	list[int16]	list[int16]	list[float]	list[float]	list[double]	list[string]	list[float]	list[float]	list[float]	int64
Nested?	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	legacysurvey	legacysurvey	legacysurvey	legacysurvey	legacysurvey	legacysurvey	legacysurvey	legacysurvey	legacysurvey	legacysurvey	legacysurvey	desi	desi	desi	desi	desi	ps1	ps1	ps1	ps1	ps1	ps1	ps1	ps1	ps1	ps1	ps1	ps1	wise	wise	wise	wise	wise	wise	wise	wise	wise	wise	wise	wise	wise	wise	wise	wise	wise	wise	wise	wise	gaia	gaia	gaia	gaia	gaia	gaia	gaia	gaia	gaia	gaia	gaia	gaia	gaia	gaia	gaia	gaia	gaia	gaia	gaia	gaia	gaia	lightcurve_yse	lightcurve_yse	lightcurve_yse	lightcurve_yse	—
Value count	10,311	10,311	10,311	10,311	10,311	10,311	10,311	10,311	10,311	10,311	10,311	10,311	10,311	10,311	10,311	15,291	15,291	15,291	15,291	15,291	15,291	15,291	15,291	15,291	15,291	15,291	10,488	10,488	10,488	10,488	10,488	27,004	27,004	27,004	27,004	27,004	27,004	27,004	27,004	27,004	27,004	27,004	27,004	10,779	10,779	10,779	10,779	10,779	10,779	10,779	10,779	10,779	10,779	10,779	10,779	10,779	10,779	10,779	10,779	10,779	10,779	10,779	10,779	10,736	10,736	10,736	10,736	10,736	10,736	10,736	10,736	10,736	10,736	10,736	10,736	10,736	10,736	10,736	10,736	10,736	10,736	10,736	10,736	10,736	37,404	37,404	37,404	37,404	10,311
Null count	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	4,997 (32.68%)	4,997 (32.68%)	4,997 (32.68%)	4,997 (32.68%)	4,997 (32.68%)	4,997 (32.68%)	4,997 (32.68%)	4,997 (32.68%)	4,997 (32.68%)	4,997 (32.68%)	4,997 (32.68%)	8,498 (81.03%)	8,498 (81.03%)	8,498 (81.03%)	8,498 (81.03%)	8,498 (81.03%)	324 (1.2%)	324 (1.2%)	324 (1.2%)	324 (1.2%)	324 (1.2%)	324 (1.2%)	324 (1.2%)	324 (1.2%)	324 (1.2%)	324 (1.2%)	324 (1.2%)	324 (1.2%)	3,981 (36.93%)	3,981 (36.93%)	3,981 (36.93%)	3,981 (36.93%)	3,981 (36.93%)	3,981 (36.93%)	3,981 (36.93%)	3,987 (36.99%)	3,982 (36.94%)	3,985 (36.97%)	4,224 (39.19%)	5,704 (52.92%)	8,029 (74.49%)	3,981 (36.93%)	3,981 (36.93%)	4,653 (43.17%)	4,653 (43.17%)	4,653 (43.17%)	4,653 (43.17%)	3,981 (36.93%)	6,462 (60.19%)	6,462 (60.19%)	6,462 (60.19%)	6,462 (60.19%)	6,462 (60.19%)	9,663 (90.01%)	9,663 (90.01%)	9,663 (90.01%)	9,663 (90.01%)	9,663 (90.01%)	9,663 (90.01%)	9,663 (90.01%)	6,463 (60.2%)	6,564 (61.14%)	6,564 (61.14%)	6,462 (60.19%)	6,462 (60.19%)	6,462 (60.19%)	9,663 (90.01%)	6,567 (61.17%)	6,462 (60.19%)	10,112 (27.03%)	10,112 (27.03%)	10,112 (27.03%)	10,112 (27.03%)	0
Example row	ZTF20aamamnp	SN2020chy	134.6	0.8746	912.7	r	18.19	-18.59	>30.957	27.957	>3	SN II	0.05	28.38	0.117	[4.43, 8.375]	[498.8, 406.4]	[7.023, 9.683]	[1090, 901.1]	[27.86, 7.153]	[33.03, 30.16]	[6.372, 10.08]	[77.27, 62.95]	[134.6, 134.6]	[0.8746, 0.8749]	[0.2641, 2.791]	[0.05257, 0.05258]	[2.163e-06, 2.523e-06]	[134.6, 134.6]	[0.8748, 0.8748]	[2.158, 2.158]	[109041346198869889, … (2 total)]	[-999, 19.92]	[-999, 0.03071]	[-999, 19.75]	[-999, 0.0336]	[-999, 19.61]	[-999, 0.04087]	[-999, 19.82]	[-999, 0.1627]	[134.6, 134.6]	[0.8745, 0.8749]	[1.494, 2.64]	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	288376908209446373
Minimum value	ZTF17aaaeqms	-	0.022458333333333327	-25.999722222222225	242.77	g	12.3536	-	0.37	0.1	0.007	-	-	-87.92203974	0.013	-3.081191062927246	-0.0	-3.4032485485076904	-0.0	-3.2418203353881836	-0.0	-87.85713958740234	-0.0	0.04430972837066446	-26.000302466888822	0.004148989161447769	-0.0005067818744003752	2.232146096148817e-46	0.20064713590971137	-18.66921481199038	0.010703883633065594	76792081881949511	-999.0	-999.0	-999.0	-999.0	-999.0	-999.0	-999.0	-999.0	0.02383356	-26.00068401	0.003955150845134975	0000m016_ac51-029488	0.0442934	-25.9998638	0.028	0.028	9.179	9.2	5.316	2.625	0.02	0.019	0.013	0.017	0.131899998	0.694899976	-14480.0	-8096.0	23.0	21.0	0.0035825245576887156	2501251514749568	0.0443097283706644	0.013420764356851578	-25.93428134785104	0.016233133152127266	-4.883082044539901	0.01960456371307373	0.008168714120984077	-35.376237166059994	0.01787719875574112	-89.35417563581815	0.018549596890807152	14.898463249206543	14.514415740966797	13.233046531677246	0	0	0	0.8511307239532471	0.7823514938354492	0.007729945139568186	g	-99.0	-0.0	-0.0	143192440943693
Maximum value	ZTF26aahaota	TDE2025vjw	359.9986666666666	87.58288888888889	3100.73	r	18.9999	-8.319	>99.92	>995.19	>99.85	other	0.519	88.64173172	0.996	17303.630859375	50353.82421875	36839.953125	30548.02734375	53510.83203125	11080.3564453125	68277.2734375	3507.349609375	359.95419757342427	35.04721559263847	4.999486778371973	3.097628040362812	0.0006964030178023236	359.6346246423098	78.51981906052832	4.998662782812608	213100167606560312	22.6077	0.33216101	22.866899	0.331487	22.941799	0.33266199	21.984501	0.33296901	359.99827034	87.58360585	4.999883924313265	3584p196_ac51-030760	359.9542415	87.3676466	1.0061	0.9319	18.391	18.243	13.77	10.173	0.485	0.541	0.542	0.543	132.399994	88.5500031	12546.0	27743.0	119156.0	133566.0	4.9987902268421935	6916208724631057280	359.84358930399	84.42621612548828	86.6580379811182	86.92155456542969	11.310966899246418	2.7759177684783936	147.4905242919922	117.342600395897	2.3149471282958984	41.15871263638486	2.8277323246002197	22.59518051147461	23.108583450317383	21.12615394592285	1494	179	177	7.945959568023682	160.05691528320312	4.999788131348823	z	59563.58984375	601750.9375	7551.6748046875	3458104365984812325

"Nested" indicates whether the column is stored as a nested field inside another "struct" column.

"Value count" may be different from the total number of rows for nested columns: each nested element is counted as a single value. "Null count" also refers to "elementary" values, not to rows.