crucible_bench

0.1.0

Comprehensive benchmarking framework for AI research. Measures latency, throughput, cost, and reliability with percentile analysis and Nx numerical computations.

HexDocs

Current section

Files

Jump to

Readme 6 Versions 2 Dependencies Dependants Files Activity

crucible_bench CHANGELOG.md

Raw

CHANGELOG.md

# Changelog

All notable changes to this project will be documented in this file.

## [0.1.0] - 2025-10-07

### Added
- Initial release
- Comprehensive statistical testing framework for AI/ML research
- Parametric tests (t-tests, ANOVA) and non-parametric tests (Mann-Whitney, Wilcoxon, Kruskal-Wallis)
- Effect size measures (Cohen's d, Hedges' g, Glass's delta, eta-squared, omega-squared)
- Power analysis with a priori and post-hoc calculations
- Confidence intervals using bootstrap and analytical methods
- High-level experiment DSL for A/B tests, ablation studies, and hyperparameter sweeps
- Publication-ready export formats (Markdown, LaTeX, HTML)

### Documentation
- Comprehensive README with examples
- API documentation for all statistical tests
- Usage examples for common AI research scenarios
- Best practices guide for statistical rigor in AI research

Operator	Description	Example
name:	Match package name. Supports * wildcards and repo/package form	name:phx* or name:hexpm/phoenix
description:	Full-text search of package descriptions	description:auth
depends:	Packages depending on a given package. Supports repo:package form	depends:ecto or depends:hexpm:ecto
build_tool:	Filter by build tool	build_tool:mix
updated_after:	Packages updated after an ISO8601 datetime	updated_after:2025-01-01T00:00:00Z
extra:	Match custom metadata (key,value). Nested keys are separated by commas	extra:license,MIT