Holl Eval

---
name: holl-eval
description: Evaluates the security resilience and hardware overhead of Higher-Order Logic Locking (HOLL) against a counterexample-guided inductive synthesis (CEGIS) attack on combinational circuits. It measures how long an attacker takes to recover the secret key relation and the area penalty incurred by the locking mechanism. Use when the user wants to benchmark on ISCAS'85 and MCNC benchmarks, or asks about evaluating this task. Reports attack_time.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2201.10531
  bibtex_key: takhar2022holl
  confidence: high
---

# holl-eval

> HOLL: Program Synthesis for Higher OrderLogic Locking — Takhar et al. (2022) (arXiv:2201.10531, 2022)

## What this evaluates

Evaluates the security resilience and hardware overhead of Higher-Order Logic Locking (HOLL) against a counterexample-guided inductive synthesis (CEGIS) attack on combinational circuits. It measures how long an attacker takes to recover the secret key relation and the area penalty incurred by the locking mechanism.

## Datasets

- **ISCAS'85 and MCNC benchmarks** — total 100; splits: test (100)

## Metrics

- `attack_time` **(primary)** — range: seconds
  - Time in seconds for the SynthAttack solver to recover a valid key relation equivalent to the original circuit. Capped at a 4-day timeout if unsuccessful.
- `area_overhead_percent` — range: percent
  - Percentage increase in circuit area due to the inserted key relation, calculated as (locked_area - original_area) / original_area * 100.
- `lock_inference_time` — range: seconds
  - Time in seconds for HOLL to synthesize the key relation, capped at a 20-minute timeout.

## Input / output format

**Input**: Combinational circuit netlists from ISCAS'85 and MCNC benchmark suites, specified by number of input/output ports and gate counts.

**Output**: Locked circuit with synthesized key relation, attack success/failure status, attack execution time, lock inference time, and area overhead percentage.

## Scoring recipe

```python
def evaluate(circuit, attack_timeout=345600, lock_timeout=1200):
    lock_start = time()
    locked_circuit, key_relation = holl_lock(circuit, budget=[12,14], depth=[2,4])
    lock_time = time() - lock_start
    if lock_time > lock_timeout: lock_time = lock_timeout
    attack_start = time()
    recovered_relation = synth_attack(locked_circuit, timeout=attack_timeout)
    attack_time = time() - attack_start
    success = (recovered_relation is not None) and equivalent(recovered_relation, key_relation)
    orig_area = get_area(circuit)
    key_area = get_area(key_relation)
    overhead_pct = (key_area / orig_area) * 100
    return {'attack_time': attack_time, 'attack_success': success, 'lock_time': lock_time, 'area_overhead_pct': overhead_pct}
```

## Common pitfalls

- Defender and attacker use asymmetric timeouts (20 mins vs 4 days), which heavily skews resilience metrics.
- Key relation complexity is constrained by a budget of 12-14 latent terms and expression depth 2-4; results do not generalize outside this range.
- Hardware overhead is measured using Nangate 15nm ASIC synthesis; direct comparison to FPGA implementations requires separate LUT estimation.

## Evidence (verbatim from paper)

> We selected 100 combinational benchmarks from ISCAS’85 and MCNC and report the time for program synthesis and the overhead after applying our locking method. SynthAttack failed to construct a valid key-relation for any of these ten designs within a timeout of 4 days. Table 4 reports the fraction of the area locked with HOLL (key relation) to the area of the original circuit.

## Citation

```bibtex
@misc{takhar2022holl,
  title={HOLL: Program Synthesis for Higher OrderLogic Locking},
  author={Takhar et al. (2022)},
  year={2022},
  note={arXiv:2201.10531}
}
```

- arXiv: 2201.10531