CNC Tolerance Checklist: Verifying Machine Limits Before Purchase

Confirm the tolerance class against your parts

Start with the actual part drawing. Identify the tightest linear dimension, hole position, and surface finish requirement. If a housing face must hold 0.05 mm, the machine’s stated CNC accuracy must comfortably beat that value, not match it. The margin exists because the part is not cut in a vacuum. It is held in a fixture, cut by a tool that wears, and measured by a probe that has its own error.

A spec sheet might list 0.002 mm repeatability. That number describes the machine’s ability to return to the same position under controlled conditions. It does not guarantee the finished part will stay within 0.002 mm after cutting, tool wear, fixture error, or thermal change. In a real shop environment, the tool diameter changes as it cuts. The workpiece expands as it heats up. The fixture may shift under the torque of the spindle.

Use the tolerance class as a filter. If the drawing calls for 5 microns, a machine rated at 10 microns is too loose. The part will fail inspection. If the drawing calls for 50 microns, a machine rated at 5 microns may be overkill and cost more than the process needs. The goal is to match the machine capability to the process requirement with enough margin to handle variation without breaking the budget.

Check repeatability versus resolution

Repeatability is the distance the machine can return to after moving to a position. Resolution is the smallest step the axis can take. These two parameters are often confused, but they measure different things. Resolution determines how finely the control can command a position. Repeatability determines how consistently the machine achieves that position.

Parameter	What it tells you	Typical red flag
Repeatability	How close the axis lands after a full cycle	Advertised value far below the measurement uncertainty
Resolution	Smallest position step	Resolution finer than the machine can actually hold
Positioning accuracy	How close the axis lands from zero	No independent verification data available

If a machine claims high resolution but weak repeatability, the control can move in tiny steps, but the result may scatter. For CNC precision work, repeatability under load matters more than nominal step size. Imagine a machine with a resolution of 0.0001 mm. The controller can command a position to that level of detail. However, if the axis has mechanical play or a worn lead screw, the tool tip may land at any point within a 0.01 mm band. The fine resolution is irrelevant because the mechanical system cannot hold the position.

Ask for the test method. A machine tested on an open floor at room temperature is not the same as one tested after a full thermal cycle. If the vendor cannot explain the test conditions, treat the number as a starting point, not a guarantee. Look for details on the reference standard used. Was the test performed with a laser interferometer or a ballbar? These methods provide different data points. An interferometer measures linear displacement, while a ballbar measures circular error. Both are useful, but they highlight different weaknesses in the machine’s kinematics.

Verify thermal stability and drift

CNC accuracy degrades when the machine warms up. A spindle, drive, or structure that changes shape by a few microns will shift the part location even if the control is perfect. Thermal growth is one of the most common reasons a machine performs well during a factory demonstration but struggles in production.

Check the thermal profile. Does the vendor provide a warm-up cycle? How long? What is the temperature change across the axis during that cycle? If the answer is vague, the risk is on the buyer. A machine that requires two hours to stabilize is not practical for a shop that runs multiple shifts with short changeovers. The thermal stability data should show the temperature change of the structure, not just the spindle. The frame, the ways, and the tool post all expand, and they may do so at different rates.

Run a simple test if possible. Move the axis to a reference point, stop for a defined time, then move again. Record the difference. Repeat after a full warm-up. If the drift exceeds your allowable tolerance, the machine needs a longer stabilization window or a more active thermal control. Some machines use active thermal compensation. This system uses temperature sensors to predict and correct for drift. It is effective, but it requires calibration. If the sensors are dirty or the algorithm is poor, the compensation can make the error worse.

Red flags include:

1. No documented warm-up procedure.
2. Drift that increases over hours instead of stabilizing.
3. Vendor claims of thermal compensation without showing the method.

Confirm fixture and workholding tolerance

The machine may be accurate, but the part may not be. A loose fixture, worn clamp, or poorly designed setup can consume the entire tolerance budget. The machine is only one part of the error chain. If the fixture is unstable, the machine’s precision is wasted.

List the workholding method for each operation. A vise with a worn jaw can add several microns of error. A fixture pin with clearance can shift the part under cutting force. Consider the specific part. If you are machining a thin-walled bracket, the clamping force must be low enough to avoid distortion but high enough to prevent movement. If you are milling a heavy gear blank, the fixture must resist the high cutting forces without flexing.

Check the tolerance of the fixture itself. If the drawing requires 0.05 mm hole position, the fixture must locate the part well inside that limit. A common rule in practice is to allocate part of the total tolerance to fixture error, tool error, and machine error. This is often called the tolerance budget. If the part tolerance is 50 microns, you might allocate 20 microns to the machine, 15 microns to the tool, and 15 microns to the fixture. If the fixture alone can vary by 25 microns, the part will fail.

If the fixture is custom, require a measurement report. If the fixture is standard, confirm the contact surfaces are flat and the clamping force is even. A machine that is accurate but mounted on a warped table will fail the final check. The floor matters too. If the machine base is not level, the axes will not move in true parallel, creating errors that compound over the length of the machine.

Measure with the right instrument

Do not verify CNC precision with a dial indicator if the tolerance is below what the indicator can resolve reliably. Use a calibrated CMM, laser interferometer, or high-precision encoder test, depending on the tolerance range. The measurement tool is as important as the machine. If you use a ruler to measure a 1 micron tolerance, the measurement is meaningless.

Tolerance range	Suitable verification method
50 microns or larger	CMM with standard probes
10 to 50 microns	High-accuracy CMM or laser system
1 to 10 microns	Interferometric measurement or high-end CMM
Below 1 micron	Specialized metrology with strict environmental control

The measurement uncertainty must be a fraction of the part tolerance. If the part tolerance is 5 microns, a measurement system with 3 micron uncertainty is not enough. The result will be noisy and unreliable. Generally, the measurement uncertainty should be one-third or less of the tolerance. This allows you to distinguish between a good part and a bad part with confidence.

Ask the vendor for the measurement report. If it is missing, request it before signing the purchase order. Do not rely on a verbal assurance. The report should state the instrument used, its calibration status, the environmental conditions, and the raw data. If the vendor uses a handheld laser distance meter for a 5 micron tolerance test, the data is worthless for that application.

Review the control and feedback system

The CNC controller interprets the machine’s movement. If the feedback loop is weak, the machine will not know when it is out of position. The controller is the brain, but it relies on the senses to see what is happening.

Check whether each axis uses encoder feedback, laser interferometer feedback, or a combination. For high-precision CNC accuracy, encoder feedback on the motor shaft is common, but a laser check at the tool tip is stronger. An encoder on the motor measures the rotation of the motor. It does not measure the position of the tool tip. If there is backlash in the gearbox or the lead screw, the motor may turn, but the tool may not move the same amount. A laser interferometer at the tool tip measures the actual displacement of the cutting tool. This eliminates the error from the transmission system.

Ask about backlash compensation. If the machine has a high-backlash drive and the compensation is only software-based, the effective accuracy may be lower than the advertised value. Software compensation works by predicting the position of the tool based on the motor position. It is effective for consistent backlash, but it fails when the backlash changes due to wear or temperature. Hardware compensation, such as a preloaded spindle or a ball screw with no backlash, is more reliable.

A red flag is a control that cannot log axis position over time. Without that data, you cannot diagnose drift, wear, or thermal effects after the machine is in production. You need the ability to export the axis position data to a computer. This data allows you to plot the position against time and see if there is a trend. If the position drifts upward over a few hours, you know you have a thermal issue. If the position jitters at a certain frequency, you may have a mechanical vibration problem.

Build the final acceptance test

Before the machine leaves the factory, run a test that matches your first production job. Cut the actual material at the actual speed and feed. Measure the result with the same method you will use in the shop. Do not accept a demonstration run with a simple aluminum block. The test part must represent the difficulty of the production job.

Define the pass criteria in writing. If the part must hold 0.05 mm, state that clearly. Do not leave the acceptance test open to interpretation. Ambiguity leads to disputes. The acceptance test should be a formal document agreed upon by both parties before the machine is built or delivered.

Include:

1. First-article inspection on the test part.
2. Dimensional check on the tightest features.
3. Surface finish measurement.
4. Repeat run of the same part after a warm-up cycle.

If the machine fails, the vendor must correct the issue before handover. If the machine passes, keep the test report as part of the machine’s documentation. The test report becomes a baseline. If the machine performs poorly in production, you can compare the current performance to the acceptance test to determine if the problem is new or if the machine was never capable of holding the required tolerance.

Red flags that should stop the purchase

Walk away if you see these issues:

1. The vendor cannot provide a calibration certificate from an independent lab.
2. The advertised CNC precision is better than the measurement uncertainty of the vendor’s own test method.
3. The machine requires a long warm-up before it reaches specification, and your production cycle cannot absorb that delay.
4. The fixture tolerance is not documented, and the workholding method is ad hoc.
5. The vendor refuses to allow a third-party verification after installation.

CNC accuracy is not a single number. It is a system of machine, control, fixture, tooling, and measurement. Verify each link before you buy.