How to Prepare Agricultural Records for AI Training

A farm record is not automatically a useful training example just because it is digital. Before records are shared with a data team or assembled into a dataset, someone should be able to answer a basic set of questions: What does this row, image or reading represent? What is the intended label? When and where was it collected? How was it measured? What is missing? Who may authorise its use?

The practical objective is not to make every file look complete. It is to retain enough context for a future user to identify records that are useful, records that need review and records that should not be used for a particular training task.

Start with the training example, not the spreadsheet

Begin with the model task rather than with the available columns. A team preparing data for an AI system should first describe the individual example the system is expected to learn from.

An example could be:

• one field observation;
• one image and its associated inspection record;
• one sensor reading over a defined interval;
• one plant, sample or plot record;
• one delivery, grading or transaction record.

These are not interchangeable. A photo may be linked to a visible condition at the time it was taken. A sensor record may describe an instrument measurement rather than a crop outcome. A harvest record may reflect a later result influenced by multiple earlier conditions. Combining them without a clear definition can create a dataset whose rows have the same format but do not mean the same thing.

The Open Geospatial Consortium’s Testbed-18 training-data work distinguishes between the example presented to a machine-learning process and the label assigned to that example. It also identifies label accuracy and consistency as important dimensions of training-data quality. That distinction is useful for agricultural operations: the record used as input and the conclusion attached to it should be separately identifiable.

ACA practical analysis: write one sentence for each proposed example type before selecting data. For example:

“One example is a dated field observation for a defined management unit, paired with an observed condition label assigned using the recorded inspection method.”

This is a working definition, not a universal data standard. Its value is that it exposes unresolved questions early. If the team cannot define the unit of observation, the target label or the time relationship between them, it should not assume the records are ready for training.

The CGIAR data harmonization guidance similarly emphasises collecting feasible descriptors that help data be interpreted across locations, time and management conditions. Complete harmonisation may not be possible, but partial alignment with clear documentation is more useful than an unexplained mixture of records.

Make labels explicit and check that they mean the same thing

A label is the answer, category or outcome attached to a training example. It might identify a crop type, an observed field condition, a management event, a quality grade or another defined outcome. Labels should not be treated as self-explanatory merely because their column names are familiar.

For every label field, record:

A simple label dictionary can prevent avoidable confusion. For example, a category called observed_status should not silently mix a worker’s direct observation, a later interpretation from imagery and a status inferred from a separate business record. Those may all be useful, but they are different kinds of evidence.

A practical label dictionary might contain entries such as:

• Field: condition_class
• Meaning: Condition category assigned to the observation unit.
• Allowed values: confirmed, suspected, not observed, unclear.
• Assignment method: Visual field inspection recorded at collection.
• Do not use for: Labels inferred later from yield records or remote interpretation.

This is a template, not a tested agricultural classification system. The correct categories depend on the intended task and the collection process.

The agricultural data-management review published through PMC discusses the challenge of annotation and semantic standards for AI and machine-learning use. In operational terms, semantic consistency means a later user should not need to guess whether the same word refers to the same condition in two files.

Where reviewers disagree, preserve that disagreement rather than forcing an artificial consensus without a record. A dataset can include a review status, alternative assessment or “unclear” category. Removing uncertainty from the file may make a table tidier while making the training signal less trustworthy.

Keep units, dates and locations attached to each observation

A value without context is often difficult to interpret. A number may be a measurement, a score, a count, a percentage or a code. A location may refer to a farm, a field, a plot, a storage site or an administrative area. A date may be the collection date, upload date, inspection date or later correction date.

For each observation type, prepare a metadata sheet that identifies the contextual fields needed to interpret it. Depending on the record, these may include:

• the unit of measurement and whether the stored value is original or converted;
• date and time of collection where timing matters;
• a location reference appropriate to the operating purpose;
• crop, production, management or sample context where recorded;
• device, form, method or process used to generate the record;
• the identity or role of the collector where access controls permit this to be recorded.

Retain original values where feasible. If a value has been converted, standardised or recoded, document the transformation and keep a link to the source record. A later analyst may need to understand whether differences arise from field conditions or from a change in units, devices or collection workflow.

The CGIAR guidance supports the use of core descriptors that allow results to be interpreted across settings. The PMC review likewise notes the need for metadata about methods and the environmental or experimental conditions in which data were generated.

A usable location reference does not necessarily mean that every shared record needs precise coordinates. A field identifier, managed block code or controlled location category may be more appropriate for some handovers. The decision should reflect the model task, access arrangement and sensitivity of the information. Location precision should be treated as an access and context decision, not as a default requirement for every farm record.

FAO material on metadata quality assurance lists items such as unit of analysis, geographic coverage, collection dates, sampling strategy, confidentiality and access conditions among the information that should be clear and complete in a documented dataset. That quality-assurance material is a useful documentation reference, not a local operating rule for Chinese farms or agricultural businesses.

Treat missing values and corrections as information

Blank cells can have many meanings: not measured, not applicable, unavailable, withheld, lost during transfer or not yet reviewed. Replacing all blanks with zero, a default category or an assumed estimate can erase information that a future user needs to evaluate suitability.

Before training, review records for:

• missing values and any available reason for absence;
• impossible or out-of-scope values identified by the collection team;
• duplicated records or repeated uploads;
• inconsistent units, date formats or category names;
• corrections made after the original record was created;
• derived fields that should not be mistaken for direct observations.

Keep three layers separate where practical: original records, cleaned records and derived fields. The original file preserves what was received. The cleaned file applies documented corrections or standardisation. Derived fields contain calculations, classifications or joined information created for analysis.

ACA recommends a simple cleaning log as an operating tool:

This is not a formal standard and it does not guarantee that a model will perform well. It does make later review more practical. A receiving team can see what was changed, what remains uncertain and which outputs are derived rather than observed.

The PMC article highlights the role of standardised data and metadata in making records understandable to people and machines. Standardisation should improve interpretability, not conceal the history of the data.

Record provenance, permissions and sharing limits

Technical quality and permission to use data are separate questions. A well-labelled record may still be unsuitable for sharing if its origin, consent conditions or contractual limits are unclear.

For each dataset or collection batch, maintain a provenance record covering:

• who collected, supplied or created the records;
• the collection arrangement or project under which they were gathered;
• the intended original use where known;
• consent, contractual or access terms relevant to reuse;
• identifiers that need removal, restriction or controlled access;
• the person or team authorised to approve a transfer;
• limits on onward sharing, aggregation or retention.

The ODK Agriculture Testing Guide provides an example of collecting information about how farmer consent was obtained because that information can affect later sharing. This illustrates an important operational principle: consent and data-sharing conditions should be recorded at collection or clarified before transfer, rather than reconstructed after files have circulated.

The Australian Farm Data Code sets out principles including transparency, control, portability and security, with attention to communicating data collection, use and sharing terms. It can help teams frame questions for a data-sharing discussion.

Neither this Australian code nor the ODK guide establishes legal requirements in China. They are not substitutes for applicable Chinese law, local privacy obligations, sector-specific requirements, farm agreements or customer contracts. Where records include personal, business-sensitive, geospatial or commercially restricted information, obtain appropriate local legal and contractual review before external sharing.

Create a dataset handover pack before model training

Do not hand over a folder of files without an explanation of what it contains. A small dataset handover pack can reduce avoidable rework and help a receiving team assess whether records fit the proposed task.

Prepare the following before model training begins:

• Data dictionary: field names, meanings, formats, units and allowed values.
• Label guide: label definitions, assignment methods, ambiguous cases and review status.
• Original and cleaned files: clearly named and kept distinct.
• Change log: corrections, exclusions, conversions and derived fields.
• Metadata sheet: collection period, location reference, methods, devices and production context where available.
• Known limitations: missing batches, different collection methods, uncertain labels and non-comparable records.
• Access conditions: permissions, restrictions, retention expectations and authorised contacts.
• Responsible contact: the team that can answer questions about collection and interpretation.

Before a full transfer, send a small review sample with its documentation. Ask the receiving team to describe what they believe each example represents, which label they would use and what limitations they see. This is a practical check of shared understanding, not a model-validation exercise.

Records from different farms, seasons, devices or management systems can sometimes be assembled in one dataset, but matching column names alone do not establish comparability. Preserve the differences in collection context and state where alignment is incomplete. As CGIAR’s guidance suggests, documented partial alignment can still be useful when full harmonisation is not feasible.

Questions to resolve before sharing

What is the difference between farm data and training data for AI in agriculture?

Farm data are records created for operational, commercial, compliance or management purposes. Training data are records selected and prepared for a defined machine-learning task. Training use requires an explicit example definition, labels, context and permission to use the records for that purpose.

Should missing values be deleted before agricultural records are used for AI training?

Not automatically. First identify what the missing value means and whether it affects the intended task. Preserve the original record, document exclusions or changes, and avoid treating all blank values as equivalent.

What metadata should accompany agricultural training data?

The necessary metadata depends on the task, but usually includes the unit of observation, label definitions, collection date, location reference, units, method or device, production context where available, known limitations and access conditions.

Can records collected on different farms be combined into one AI training dataset?

They can be considered together, but they should not be assumed directly comparable. Review differences in labels, units, dates, location references, collection methods, devices and management conditions. Record those differences even when the files are combined.

The next step is bounded: choose one proposed AI use case, take a small sample of its records and create a one-page example definition plus a label dictionary. If that sample cannot be explained clearly to another team, expand the documentation before investing in large-scale cleaning or model development.