What does it actually mean for a machine to learn?

The question

A model gets every training example right. Has it learned anything useful?

Imagine estimating delivery time. Your training table contains distance, traffic, and the actual arrival time for each trip. A lookup table can remember every answer. But the next trip will not be identical to the last one. The useful question is whether the system can make a good prediction for a trip it has never seen.

Learning is an optimization problem

Choose a family of functions, a way to measure error, and an algorithm that adjusts the function’s parameters. Training uses examples to reduce that error. Inference applies the fitted function to a new input.

For a simple linear model, predicted minutes might equal a learned constant plus a learned coefficient multiplied by distance. Training changes those two numbers. It does not install a general understanding of roads, weather, or human intentions.

The test is outside the training set

Hold out examples before you fit preprocessing or the model. Use validation data to choose settings. Keep the final test set separate from those repeated decisions. Cross-validation can help estimate variability, but the split must reflect the application.

For delivery times, a random split may be misleading if adjacent trips share traffic conditions. A time-based split asks a more realistic question: can a model trained on the past predict a later period? Scikit-learn documents these evaluation and model-selection distinctions in its official guide.

A concrete failure

Suppose the table includes a field recorded only after the delivery finishes. A model might predict brilliantly by exploiting that field. This is leakage: information that would not exist when a real prediction is needed.

Write down the prediction moment. For every input, ask whether it is available at that moment. Repeat the exercise for transformations, aggregations, and labels, not only raw columns.

Choose the error that matters

Average error can conceal rare but expensive failures. In our example, an estimate that is consistently five minutes optimistic may frustrate customers more than one with the same average absolute error but no directional bias.

Report a simple baseline, overall error, and errors for meaningful slices such as long-distance trips. Do not choose slices only after discovering where your model looks impressive.

What comes next?

Before selecting a larger model, investigate the examples it gets wrong. Is the target noisy? Is an input unavailable? Has the environment changed? A useful learning system includes that investigation, not just the fitting step.

Try it: design a delivery-time table with five inputs. Mark when each input becomes available. Then choose a train/test boundary and explain which future situation your test actually represents.