Installs into .claude/skills of the current project.
Are you the author of Dataframes Tfrecords?
Add the live security badge to your README. It updates with every re-scan.
[](https://www.skillsdirectory.com/skills/vectorspacelab-dataframes-tfrecords)
---
name: dataframes-tfrecords
description: "Route Spark DataFrame to/from TFRecord conversion, schema
inference, and MNIST row-shaping requests."
disable-model-invocation: true
metadata:
disco-role: operating
license: Apache 2.0
---
# DataFrames and TFRecords
Use this sub-skill when the request is about:
- `dfutil.saveAsTFRecords`
- `dfutil.loadTFRecords`
- `dfutil.toTFExample`
- `dfutil.infer_schema`
- `dfutil.fromTFExample`
- `binary_features`
- TFRecord Hadoop classpath setup for Spark DataFrames
- the small MNIST CSV row reshaping helper
## What it covers
- Flat Spark DataFrame ↔ `tf.train.Example` conversion
- Spark SQL type mappings for scalars, primitive arrays, strings, and raw bytes
- The TensorFlow Hadoop `TFRecordFileInputFormat` / `TFRecordFileOutputFormat` dependency
- Small MNIST row formatting for `saved_model_cli` or serving-style input examples
## Route elsewhere
Do not use this sub-skill for:
- TensorFlowOnSpark training or inference data feeds
- Spark ML pipeline modeling or estimator workflows
- full MNIST downloads or training recipes
## Start here
1. Read [API reference](references/api-reference.md) for function contracts and signatures.
2. Read [data formats](references/data-formats.md) for supported Spark/TFExample mappings.
3. Read [troubleshooting](references/troubleshooting.md) when classpath, bytes, or dtype errors appear.
4. Run [schema probe](scripts/tfos_tfrecord_schema_probe.py) for a tiny local smoke test.
5. Use [MNIST reshape helper](scripts/mnist_reshape.py) when a single CSV row must become a `[28, 28, 1]` literal.
## Expected signals
- `saveAsTFRecords` writes TFRecord files through the TensorFlow Hadoop output format.
- `loadTFRecords` reconstructs a DataFrame schema from the first serialized example.
- `binary_features` must be supplied whenever a `BytesList` should remain raw bytes instead of UTF-8 text.
- Unsupported Spark dtypes should be normalized before conversion.
- Missing TensorFlow Hadoop classes should be fixed on the Spark classpath before retrying.
## Boundaries
If the user needs model training, inference, or full example conversion, keep the request outside this sub-skill and hand it to the appropriate higher-level sub-skill.
