Tolerance is a drawing number; capability is a distribution you can measure

A lot of 0.8 mm septa arrived at our dock in 2019 with a certificate claiming a 1.6 mm puncture corridor and a durometer of 45 Shore A. We sampled 30 from the carton with a Mitutoyo digital micrometer and got a thickness RSD of 4.1% across the lot, with a max-min spread of 0.11 mm. That is not a cosmetic problem. On a 7890 inlet with a 23-gauge needle, a 0.11 mm thickness swing pushes the needle tip past the septum seat on one in twelve injections, and the resulting coring shows up as a split peak on the fourth injection of a sequence. The certificate was accurate for the sample the molder sent. It was not accurate for the lot we received. That gap between a certificate and a process is what “dimensional tolerance capability” actually means, and it is the subject of this article.

If you are sourcing septa, ferrules, vial caps, column hardware, or filter housings, you have probably been quoted a tolerance and assumed it was a process capability. In lab chromatography dimensional tolerance capability, the number on the drawing and the number the process can hold are two different things. One is a promise. The other is a measurement, and they only match when someone has done the work to prove it.

A drawing tolerance of ±0.05 mm tells you what the designer will accept. It says nothing about what the molding process will produce. Capability is the relationship between the specification limits and the natural spread of the process, expressed as Cpk (or Ppk if you are using long-term data). Cpk is the smaller of (USL minus mean) divided by 3 sigma, or (mean minus LSL) divided by 3 sigma. A Cpk of 1.33 means the process spread fits inside the tolerance band with about 25% margin on each side. A Cpk of 1.0 means the process is just barely inside the limits, and roughly 2,700 parts per million will fall outside. A Cpk of 0.8 means you are shipping defects and calling them conforming because your AQL sampling plan does not catch them.

For lab consumables, I use these thresholds as working rules:

  • Cpk ≥ 1.67 for any dimension that touches the fluid path or seals against it (septum thickness, ferrule bore, vial neck ID, filter housing seat).
  • Cpk ≥ 1.33 for dimensions that affect fit but not sealing (cap skirt OD, column end-fitting thread major diameter).
  • Cpk ≥ 1.00 is a red flag on any dimension the customer can measure with a caliper. If they can measure it, they will, and they will find your tail.

The capability number is only meaningful if it is calculated from a stable process. Run a control chart on the dimension first. If the mean drifts across the lot (tool wear, mold temperature creep, resin lot change), the Cpk is fiction. I have seen suppliers quote Cpk of 1.5 from a 30-piece study taken in the first hour of a run, then ship a lot with a 0.08 mm mean shift because the mold cooled unevenly over eight hours. The study was real. The process was not stable.

What each process can actually hold, by material and feature size

The tolerance a molding process can hold depends on the material, the wall thickness, the feature geometry, and the tool. Here is what I have seen hold up in production, not in a sales brochure:

Process Typical achievable tolerance Best-case on a well-controlled feature Where it breaks down
Injection molding, PP or PE, feature < 25 mm ±0.10 mm ±0.05 mm on a single cavity with a stable tool Multi-cavity tools; cavity-to-cavity variation adds 0.03 to 0.08 mm
Injection molding, PTFE or PFA ±0.15 mm ±0.08 mm with a heated tool and slow cycle Shrinkage varies with sinter time and resin lot; post-mold creep
Compression molding, silicone septum ±0.10 mm on thickness ±0.05 mm with a ground cavity and controlled cure Flash line thickness, cure shrinkage, demolding distortion
Machined PTFE (ferrules, housings) ±0.05 mm ±0.02 mm on a CNC lathe with temperature control Tool wear, thermal expansion of the workpiece (PTFE moves ~0.12 mm per 100 mm per 10 °C)
Machined PEEK (column hardware) ±0.025 mm ±0.01 mm Moisture absorption before machining; stress relief after
Extruded tubing (PTFE, FEP) ±0.05 mm on ID ±0.02 mm on a calibrated line Draw-down variation, die swell, cooling rate

Two things to notice. First, the material matters more than the process label. A machined PTFE part can hold ±0.02 mm on a lathe, but if the shop does not control shop temperature, the part measures differently at 9 a.m. and 3 p.m. Second, multi-cavity tools are where capability quietly dies. A 16-cavity septum tool can have a cavity-to-cavity thickness spread of 0.06 mm even when each cavity is individually capable. If your supplier quotes a single Cpk without telling you the cavity count, ask. The number is probably the best cavity.

For reference on what the major instrument makers publish for their own consumables, the Agilent GC columns and accessories catalog and the Thermo Fisher chromatography and sample handling pages both list dimensional specs on septa, ferrules, and liners. Those numbers are worth reading as a benchmark for what an OEM considers a shippable tolerance, not as a substitute for your own capability study.

The dimensions that are genuinely hard to control

Not all dimensions are equal. Some are easy to hold within ±0.02 mm and some fight you every cycle. The hard ones, in my experience:

Septa thickness and puncture corridor

A septum is a soft, compressible part. Thickness is easy to measure with a micrometer but hard to hold because the material compresses under the anvil. The right way to measure is with a flat anvil and a consistent applied force (I use a constant-force micrometer at 5 N). The wrong way is a caliper, which will read 0.02 to 0.04 mm thin because the jaws pinch. The puncture corridor (the region the needle passes through without hitting the seat) is a derived dimension: it depends on thickness, durometer, and the geometry of the inlet. A septum with a nominal 1.6 mm thickness and a 45 Shore A durometer will deflect about 0.15 mm under a 23-gauge needle at 2 N insertion force. If your inlet seat clearance is 0.10 mm, you are already interfering.

Ferrule bore and cone angle

A graphite or Vespel ferrule that seals a 0.25 mm ID column has a bore tolerance of maybe ±0.03 mm and a cone angle tolerance of ±0.5°. The cone angle is the killer. A 1° error over a 6 mm cone length moves the sealing face by about 0.1 mm, which is enough to cause a leak at 20 psi inlet pressure. I have seen ferrules that passed the bore check and failed the cone angle check, and the leak only showed up on a pressure decay test at 30 psi, not on the bench. If your supplier does not measure cone angle, they are not measuring the dimension that matters.

Vial neck ID and crimp geometry

A 2 mL crimp vial has a neck ID of about 6.0 mm and a crimp skirt that must seat within ±0.15 mm of the neck shoulder. If the neck ID runs 0.1 mm large, the septum sits lower and the crimp does not compress it enough, which shows up as a leak on the autosampler. If the neck ID runs 0.1 mm small, the septum bulges and the needle cores it. The COD vial line is a good example of a product where the neck and crimp dimensions are the whole game, and where a 0.1 mm drift is the difference between a clean injection and a sequence full of split peaks.

Filter housing seat and membrane retention

A syringe filter housing has a seat that holds the membrane in place under pressure. The seat depth tolerance is typically ±0.05 mm, but the membrane thickness itself varies by ±10% on a PTFE membrane. If the seat is too deep, the membrane shifts and you get channeling; if too shallow, the housing cracks at 100 psi. The HPLC filter and PTFE membrane filter product lines are where I see this most often, because the membrane and the housing are made by different processes and the tolerance stack has to be managed across both.

How to measure capability without a statistician

You do not need a Six Sigma black belt to run a capability study. You need a stable process, a calibrated gauge, and 30 to 50 parts taken across the run, not from the first hour. The steps I use:

  1. Define the dimension and the gauge. Write down the measurement method, the gauge, the applied force, and the temperature. A micrometer reading at 22 °C is not the same as one at 28 °C for a PTFE part.
  2. Take 30 to 50 parts across the lot. Not from one cavity, not from one hour. If the tool has 16 cavities, take at least 3 parts from each cavity.
  3. Plot the data. A histogram and a run chart. If the run chart trends, stop. The process is not stable and Cpk is meaningless.
  4. Calculate mean and standard deviation. Cpk = min((USL – mean)/(3σ), (mean – LSL)/(3σ)). Use the sample standard deviation, not the range.
  5. Check the gauge R&R. If your measurement system contributes more than 10% of the tolerance, your capability number is measuring your gauge, not your process. A micrometer with a worn anvil can add 0.02 mm of variation all by itself.

For septa, I use a 30-piece sample, a constant-force micrometer at 5 N, and a 22 ± 1 °C room. For ferrules, I use a pin gauge for bore and an optical comparator for cone angle. For vials, I use a go/no-go gauge on the neck and a crimp-height gauge on the finished crimp. The point is to measure the dimension that fails in the field, not the one that is easy to measure.

Two field cases where the tolerance stack told the story

Case 1: the septum that passed the certificate and failed the inlet

A pharma customer reported split peaks on the fifth injection of every sequence, with a retention time RSD of 2.8% on a method that normally held 0.4%. The first suspect was the column. The second was the inlet liner. Both were fine. We pulled the septum from the inlet and measured it: 1.72 mm thick, against a nominal 1.60 mm. The certificate said 1.60 ± 0.05 mm. The lot had drifted because the molder had changed cure time to hit a delivery date, and the longer cure shrank the part less. The fix was not a new septum supplier; it was a gauge on the dock. We added a 5 N constant-force micrometer check on every incoming lot, with a 30-piece sample and a reject limit at 1.65 mm. The split peaks went away, and the next three lots came in at 1.61 ± 0.02 mm because the supplier knew we were measuring.

Case 2: the ferrule cone that leaked at 30 psi

A customer running a GC method with a 0.25 mm ID column at 25 psi inlet pressure started seeing a slow pressure decay on the leak check, about 0.5 psi per minute. The ferrule was graphite, nominal cone angle 15°, bore 0.5 mm. We measured 40 ferrules with an optical comparator and found a cone angle spread of 1.2°, with 6 of 40 outside ±0.5°. The bore was fine on all 40. The supplier had been checking bore with a pin gauge and assuming the cone was controlled by the tool. It was not; the tool had worn. The fix was a cone-angle gauge on the supplier’s line and a 100% leak test at 30 psi on our dock for that part number. The leak rate dropped to zero on the next lot.

A decision checklist for evaluating a supplier’s tolerance claim

When a supplier quotes a tolerance, ask these questions before you put the part on a validated method:

  • What is the Cpk, and on which cavity? If they cannot answer, the number is a drawing tolerance, not a capability.
  • How many parts, over what time window? A 30-piece study from the first hour is not a capability study.
  • What gauge and what force? For soft parts, the measurement method is part of the tolerance.
  • What is the cavity-to-cavity spread? A 16-cavity tool with 0.06 mm spread needs a tighter per-cavity target to stay inside the overall spec.
  • What is the gauge R&R? If it is above 10% of the tolerance, the capability number is noise.
  • What is the incoming inspection plan? A certificate is not an inspection. Ask for the sampling plan and the reject limit.

For context on how these consumables fit into a regulated method, the U.S. EPA SW-846 test methods and the Restek technical resources both describe the dimensional and performance requirements that a method assumes. Those documents are written for the analyst, not the molder, but they tell you which dimensions the method actually depends on. The Lab Chromatography product pages are a useful cross-reference for what a given part number is supposed to measure.

The rule I use when a tolerance claim arrives without a capability study

Treat every tolerance on a drawing as a hypothesis until you have seen a control chart, a gauge R&R, and a 30-piece sample taken across a full run. If the supplier cannot produce those three things, buy the part on a trial basis, measure the dimension that fails in the field on your own dock, and set your own reject limit. The drawing number is where the conversation starts. The capability number is where the purchase decision gets made, and it is the only number that tells you whether the lot in the carton will behave like the sample in the certificate.

References & further reading

Related resources from our lab consumables network:

External references and standards: