Choosing fairly when you don't know how many you're choosing from.
A rational analysis of the selection task as optimal data selection.
A rational analysis of the selection task as optimal data selection.
#Exploration: A Study of Count-Based Exploration for Deep...
Count-based exploration algorithms are known to perform near-optimally when used in conjunction with tabular reinforcement learning (RL) methods for solving small discrete Markov decision...
Beware of samples! A cognitive-ecological sampling approach to judgment biases.
PFAS Sampling Rates in Microporous Polyethylene Tube Samplers
<p><strong>Dataset Description</strong></p> <p> This dataset contains PFAS water concentrations measured using grab samples and PFAS accumulated in microporous polyethylene tube (MPT) passive samplers across five field campaigns. These data were used to reevaluate PFAS sampling rates for MPT passive samplers. An overview of sampling sites and sources of temperature data used in the reevaluation is also provided. </p> <p> The five field campaigns are labeled according to principal investigator and field site: </p> <ul> <li><strong>CG_RI</strong>: Christine Gardiner, Narragansett Bay, Rhode Island</li> <li><strong>JS_DU</strong>: Jarod Snook, Duluth, Minnesota</li> <li><strong>JS_ME</strong>: Jarod Snook, Maine</li> <li><strong>MD_CC</strong>: Matthew Dunn, Cape Cod, Massachusetts</li> <li><strong>MD_WE</strong>: Matthew Dunn, Westerly, Rhode Island</li> </ul> <p> For each field campaign, a README file is included that provides the information necessary to locate, interpret, and reuse the associated data. For studies conducted by Jarod Snook, raw data files are provided directly. </p> <p> Cleaned and compiled datasets are provided as <em>grab_samples.csv</em>, containing data from grab samples, and <em>pe_tube.csv</em>, containing data from MPT deployments, grab samples, and re-evaluated sampling rates. </p> <p> Compound and water type specific PFAS sampling rates derived from averaged values and multiple fitting approaches (linear, inverse linear, and exponential) are provided alongside reference sampling rate data from Australia and Spain in <em>parameters_fresh_salt.csv</em> and <em>parameters_ground.csv</em>, respectively. </p>
Running ML models in a game (and in Wasm!)
Using Tract and Bevy, guess the number being drawn.
Opinion | This Is What Will Ruin Public Opinion Polling for Good
Instead of navigating the obstacles to conduct polls with human respondents, pollsters are running A.I. simulations instead. Why?
Adding Realistic Rivers to Random Terrain
Discusses a method for adding rivers to randomly generated terrain that adds a realistic touch.
Complete guide to samplers in Stable Diffusion
Dive into the world of Stable Diffusion samplers and unlock the potential of image generation.
PFAS Statewide Sampling Dataset: PFAS drinking water sampling data from U.S. community water systems
This repository contains a sample of U.S. community water systems (CWS) included in the PFAS statewide sampling dataset. In the first versions of this dataset, 18 states were included as part of Liddie et al., 2023. The statewide sampling dataset now includes compiled data from 28 U.S. statewide sampling campaigns of CWS for per- and polyfluoroalkyl substances (PFAS) in drinking water. The full dataset of samples spans the 2016-2024 period. All data were either publicly available or provided by state agencies upon request. The full dataset contains approximately 98,000 samples from 11,138 CWS in Alabama, Arizona, California, Colorado, Georgia, Illinois, Indiana, Iowa, Kentucky, Maine, Maryland, Massachusetts, Michigan, Minnesota, Missouri, New Hampshire, New Jersey, New York, Ohio, Pennsylvania, Rhode Island, South Carolina, Tennessee, Utah, Vermont, Virginia, Washington, and Wisconsin. Concentrations of 5-40 targeted PFAS were analyzed by multiple labs contracted by the states using comparable methods, including either US EPA Method 537, 537.1, or 533. We assigned a conservative, uniform detection limit at the 98th percentile of all reports detection limits (5 ng/L) for the 5 PFAS that were measured in all states consistently. This dataset also incorporates data from the US EPA’s Safe Drinking Water Information System, including the source water type, county served, zip code, system city, and total population served. The 8-digit hydrological unit code in which a CWS intake is estimated to reside is also provided using data from the United States Geological Survey. This was done by geocoding each CWS using geographical data provided by state agencies, reported zip codes (when available), or based on additional information provided by each system (e.g., system name, city, city served). At this time, the full datasets (either system-level or sample-level) are available by request (see contact information below). Note that additional fields, such continuous concentration data, reported detection limits, concentrations of other PFAS analytes, are also available. Proper citation of both this repository and the original associated paper (see recommended citations below) are required for use of these data. Due to the significant time investment required to curate these data, additional data requests beyond the scope of the details listed here are welcomed but should be agreed upon (e.g., in terms of co-authorship) with the primary point of contact. Please contact Jahred Liddie (jliddie@g.harvard.edu) for questions/suggestions, access to additional variables, or access to the full sampling datasets.
AI Aqueducts: Speed. Confidence. And Contaminants.
The AI Pipeline Has No Way to Judge the Science It Uses
LAVA: Data Valuation without Pre-Specified Learning Algorithms
Traditionally, data valuation is posed as a problem of equitably splitting the validation performance of a learning algorithm among the training data. As a result, the calculated data values depend on many design choices of the underlying learning algorithm. However, this dependence is undesirable for many use cases of data valuation, such as setting priorities over different data sources in a data acquisition process and informing pricing mechanisms in a data marketplace. In these scenarios, data needs to be valued before the actual analysis and the choice of the learning algorithm is still undetermined then. Another side-effect of the dependence is that to assess the value of individual points, one needs to re-run the learning algorithm with and without a point, which incurs a large computation burden. This work leapfrogs over the current limits of data valuation methods by introducing a new framework that can value training data in a way that is oblivious to the downstream learning algorithm. Our main results are as follows. $\textbf{(1)}$ We develop a proxy for the validation performance associated with a training set based on a non-conventional $\textit{class-wise}$ $\textit{Wasserstein distance}$ between the training and the validation set. We show that the distance characterizes the upper bound of the validation performance for any given model under certain Lipschitz conditions. $\textbf{(2)}$ We develop a novel method to value individual data based on the sensitivity analysis of the $\textit{class-wise}$ Wasserstein distance. Importantly, these values can be directly obtained $\textit{for free}$ from the output of off-the-shelf optimization solvers once the Wasserstein distance is computed. $\textbf{(3) }$We evaluate our new data valuation framework over various use cases related to detecting low-quality data and show that, surprisingly, the learning-agnostic feature of our framework enables a $\textit{significant improvement}$ over the state-of-the-art performance while being $\textit{orders of magnitude faster.}$
