evaleval/benchmark-saturation

AI Summary: This issue requests the implementation of a new metric, `InputSampleDomain`, which describes the domain of input samples (e.g., coding) for datasets listed in `dataset_hf_id.csv`. The goal is to automate its extraction via web scraping, with manual data entry as a fallback, by adding a new function to an existing class in the `analyzer` branch.

AI Summary: The task involves migrating PCA computation from the browser to a Python backend. This includes removing the client-side PCA code, creating a Python script to perform dimensionality reduction (e.g., t-SNE) on benchmark data, and updating the website to load and visualize the precomputed results.

AI Summary: Improve the visualization of benchmark clustering by incorporating relevant properties like saturation status (saturated/unsaturated) to enhance the interpretability of the current clustering results. This involves finding a suitable visualization method to represent these properties effectively, potentially using color-coding or other visual cues.

AI Summary: Improve the website loading time by implementing a loading indicator (ideally with a progress bar) and optimizing the JavaScript bundle size through tree shaking. Explore additional optimization strategies such as metadata compression, if feasible.

AI Summary: Develop a filtering dashboard to replace an existing overview table for a benchmark saturation project. The dashboard should visualize benchmark saturation, link to corresponding tasks, allow sorting/filtering based on various criteria (e.g., from provided mock data), and dynamically update aggregate visualizations (currently only PCA). This likely involves multiple pull requests and will require choosing a suitable library (DataTables suggested).

AI Summary: Clean up and restyle the project's landing page to match the EvalEval website's style. This involves removing outdated content, improving code quality (TypeScript and HTML), and reorganizing the code for better maintainability and future development.

AI Summary: The task involves aggregating and visualizing statistics from datasets used in AI benchmarks. This includes brainstorming additional relevant properties beyond existing PCA analysis and implementing the visualization of these properties. The goal is to provide a quick overview of dataset characteristics to better understand benchmark saturation dynamics.