Custom cybersecurity datasets
We develop high-difficulty tasks to improve models’ cybersecurity capabilities. We directly review whether models genuinely struggle with each task and whether its difficulty has been artificially increased.
With human expertise,
we advance AI.
CYVARC is a team of top cybersecurity experts. We create and verify high-difficulty tasks to improve AI’s cybersecurity capabilities.
ReliabilityWe personally review every task by hand.
High qualityOriginal, fair tasks that experts can solve.
CustomWe create tasks that meet your goals and format.
We build datasets to improve models’ cybersecurity capabilities,
and establish criteria for choosing which AI products to use.
Benchmarks focused on specific capabilities
Every company has different goals and a different vision. AI-based products are multiplying, but there is no tailored assessment to help companies decide which product to use. We develop specialized benchmarks around a company's goals and vision, creating assessments that quantify which AI product is the best fit.
AI products
Guardrail compliance during work94%
Time to apply user instructions↓1.2 s
Agent action traceability72%
Detection result consistency84%
Target workflow completion rate56%
Product requirements
Automate vulnerability checks for your services. More important than finding high-impact vulnerabilities is enabling people to quickly control and intervene in important actions.
One task. Three uses.
- Service environment
- Agent instructions
- Code-based static grader
- Reference solution
- Solution explanation
RL
Define criteria for fair rewards
SFT
Detail experts’ reasoning processes
EVAL
Test sets for fair performance evaluation
Simply having more data
does not improve a model’s capabilities.
Data is one of the only ways to teach a model. Models trained on bad data have much lower performance and capabilities than models trained on good data. Just as excellent teachers develop excellent students, good data maximizes a model’s performance and capabilities.
Bad data
Unfair, artificial tasks that cause models to fail for the wrong reasons
The task blocks a fair solution.
- Problems solvable only through excessive guessing
- Internet-dependent solutions
- Excessively trick-based problems
- Unreasonable evaluation logic
- Time-dependent, race-against-the-clock problems
Solving the task requires uncovering an unstated, unreasonable assumption.
Good data
Fair tasks that models find difficult for valid reasons
An agent can fail. An expert can solve it.
- Reasoning problems an expert can reasonably work out
- Sufficient evidence in the provided materials
- Genuine reasoning difficulty
- Relevant completion checks
- Problems that remain unchanged over time
An expert can combine the given evidence, reason from it, and derive a solution.
Hard
Even frontier models face substantial difficulty.
Honest
Tasks must be solved through reasoning, not guesswork.
Solvable
Skilled experts can solve them themselves.
External data.
Contains a large amount of bad data.
More content also means more noise. The internet and cybersecurity datasets currently being delivered contain errors, duplication, and artificial difficulty.
Internet corporaSupplied datasets
Good data is easy to imitate.
Selecting it and creating good datasets requires expertise.
Security expertise alone is not enough.
CTF and practical expertise alone do not produce training-ready data. But CYVARC takes responsibility for creation standards and review, so clients do not need to spend additional effort on quality.
With domain experts alone
Client requirements are not met
Although a domain expert created it
Repeated revisions are needed
With CYVARC
Using criteria agreed with the client in advance
Through internal expert review
We deliver verified results
From original, high-difficulty tasks
to verified datasets.
CYVARC's experts personally create, test, and review every task. Only the few tasks that pass this process are included in the dataset delivered to the client.
- 01Client defines the scope
- 02CYVARC designs
- 03Experts create tasks
- 04Review
- 05Dataset delivery
Create original, high-difficulty tasks
Selected experts create the task, build its environment, and record both successful and failed approaches.
Test and review
Can it be reproduced at any time?
Is the difficulty fair?
Sound reasoning
Was the reward earned honestly?
Return for revisionRepair the reward check, then review again.
A second expert solves it blind, inspects agent trajectories with Tragency, and challenges the reward logic.
Select and deliver
- Verified solution
- Reviewed agent trajectories
- Expert review evidence
- Reward checks
Only approved tasks enter your dataset, with the environment, solution, grader, and supporting evidence.
Which domains do we specialize in building datasets for?
Expert reasoning
We provide original, high-difficulty tasks, verified solutions, and structured reasoning, including failed approaches.
We design fair reward structures
We design and provide graders that check the actual state of the solution, not the agent’s claim of completion.
Web security
Binary exploitation
Reverse engineering
Cryptography
Cloud security
Smart contracts
CYVARC’s rigorous standards.
- Every task hand-reviewed
- No unfair difficulty
- Revisions until acceptance
Review
- Hand review, no sampling
- Blind independent re-solve
- Reward logic verified
- Unlimited revisions
Authenticity
- No unfair difficulty
- No recycled CTF ideas
- Idea-level deduplication
Engineering
- Repeatable runs and pinned versions
- Frontier-calibrated
- Intended solution and writeup
Read all ten CYVARC commitments
Hand review, no sampling
Every task is manually reviewed by our own team. No sampling.
Blind independent re-solve
An agent solves the task again without seeing the intended solution.
Reward logic verified
We actively look for reward hacks and shortcuts.
Unlimited revisions
We keep reworking the batch until you accept it.
No unfair difficulty
No context flooding. No answers that depend on guesswork.
No recycled CTF ideas
No recycled CTF challenges or writeups.
Idea-level deduplication
We create one task from one idea. We deduplicate at the idea level.
Repeatable runs and pinned versions
Tasks can run in containerized environments with pinned versions.
Frontier-calibrated
Difficulty is calibrated against current frontier model pass rates.
Intended solution and writeup
A writeup, the intended solution, and an explanation of why the task is hard.
Exceptional hackers.
A carefully selected few.
CYVARC creates tasks together with teams from offensive security companies. Our partners have placed among the top teams in numerous international CTFs and have experience in in-depth security vulnerability research and advanced challenge creation. We work on projects with only a specially selected few from among them.
CYVARC is selective about both its creators and its review.
Only a carefully selected few participate in task creation, and an even smaller group of reviewers examines every submission against strict standards before final delivery.
Plain Co-founder
Founder, RewriteLabPlain leads AI security technology at STEALIEN, a Korean offensive security company, and researches web2 and LLMs. He conducts advanced web2 vulnerability research with international companies and is the founder of RewriteLab, a team of top-tier web security experts. At CYVARC, he oversees task quality review and operational workflow design.
Boron Co-founder
@obl1vionisBoron is an independent security researcher. He has helped secure over $10 billion in assets and discovered vulnerabilities in critical blockchain and financial infrastructure, including Ethereum and Monero.
The clients already working
with us are the proof.
Companies such as Anthropic already work with us, and their feedback has been highly positive.
You know your models.
We know cybersecurity.
If you need trustworthy cybersecurity datasets and benchmarks, we can help.
Frontier AI labs
Original cybersecurity tasks for training stronger models, with expert reasoning and reward checks grounded in genuine task completion.
Open-model developers
Fine-tuning datasets focused on the specific cybersecurity capabilities you want to improve, with tasks tailored to your model, desired domain, and difficulty.
Data-labeling companies
Specialist cybersecurity task creation and initial validation for companies that build and curate datasets across multiple domains.
Start with a sample.
See the difference.
You can receive sample tasks. See for yourself why our datasets are high quality.
Your requirements are the starting point.
- Domain
- Difficulty
- Volume
Tell us what you need. We’ll come back with a plan and a price, in a format that fits your pipeline.