Autoleveller

Hi James,

I have had success with your autoleveler. I produced my very first circuit board set. I would like to explain my use for pcb and proposed use for milling plastic and other materials, and ask for an additional feature from you, for which I am willing to donate $100 in advance.

A brief workflow description for making my first pcb follows for background:

I used Kicad to produce the layout and to tile 4 individual boards with rounded corner cutout outline onto one gerber file.

Exported gerber and excellon files from Kicad Plot output.

Opened gerber and excellon drill file in Coppercam from France, 80 Euros and a good deal for the money.

Coppercam lets me create all gcode for traces, hatching to remove excess copper between traces, drilling, and cutout of the 4 individual boards with rounded corners.
Could have used pcb-gcode, but needed Eagle to do that. It was cheaper to buy Coppercam and use Kicad for the original design.

Next comes your Autoleveler app. I open the gcode file from Coppercam using Autoleveler, set the probe depth to -2 so it dosen't give me an error for the probe movement finishing before touching, and adjust the probe set to be under 200 probes (over that amount of probes gives me an error in Linuxcnc) and within the boundaries of the pcb so the probing happens inside the mounting clamp extremities.

I had a learning curve to figure it out, but It runs and did the job well.

Now, here is my feature request:

I would like to be able to use Autoleveler in the following way, which I believe would increase its utility:

I check a box in autoleveler to make it generate code which uses the linux cnc feature PROBEOPEN (do the probes) PROBECLOSE, which saves probe data to a file instead of variables in memory.

The output in this case from Autoleveler is simply the probe routine, without any gcode to cut parts.

This routine when run on linuxcnc performs the probing grid on the cnc machine and stores the probe values in a file on the Linux cnc computer. I copy this file to my usb memory stick, and walk it back over to my laptop running Autoleveler. (Or same machine)

I open this probe data file in autoleveler using a "Open Linux cnc probe point cloud" file open feature. Mach3 probably has a similar capability to save probe data in a file, but it may be slightly different data format.

Now, I can "level" any gcode that I want by clicking "use point cloud data to autolevel g code file". This would rock. I could use a touch probe to use autoleveler for milling long workpieces of other materials, not only pcbs. Very useful for unlevel beds and long or wide workpieces with substantial error.

Also, we overcome the probe count limit imposed by storing the probe values into variables. Instead they go to a file. You can keep the original design, but allow this new mode as well. In this new way, if there is a need to rerun the milling operation, I don't have to re-probe.

This would then give me a adjusted gcode file which would just run, without the need to do the probing first. In other words, it has already been corrected in advance by the above steps. Of course the files to be leveled must match the probing rectangle, but that's fine.


I will have the money to do this in December. As a fellow software developer, I think you can keep the original code pretty much, all you are doing is using PROBEOPEN routine in gcode to create the probe data file, instead of saving it to variables in linux cnc, and creating a dialog to accept this probe data file, store it in an array in java, and use it in the normal way to produce corrected z values, and save them in the final output file. This gcode file when run in linux cnc would not begin with probing, but would simply run the pre-corrected gcode and mill the part.

PS, there may be some emails on your website "contact us" system from when I began this adventure.

Any questions or thoughts?

Hi Doug,

This is indeed a good idea. I wanted to add this feature from the start and I know this would be fairly straightforward to do in LinuxCNC. This was discussed quite recently in the thread "Forgot to mention..." in general discussion.

The biggest problem is that I have no idea how to log in mach3 as you would in linuxcnc (i.e.directly from the gcode itself with LOGOPEN, PROBEOPEN etc.). I know I will get lots of complaints from mach3 users if this feature is only present in linuxcnc so I wanted to add this feature to both.

The other problem is that I just started a new job this week, which will divert my energy away from the AutoLeveller, at least till I find my feet with this job so to speak. To be honest, I would much rather work solely on the AutoLeveller and other CNC software but it does not guarantee an income. At least not yet.

In the meantime, the previously mention thread discusses about data persistence and adding variables you want saved in the file "src/emc/rs274ngc/interp_array.cc". You could probe once then delete the probing part for any subsequent mills manually. Its not ideal but it might be worth looking into for now.

Fine business. Good luck on your new job. I have done some basic research on how to do this in Mach3 in an attempt to save you time getting this feature together. Perhaps this information would narrow the problem down enough for you to be enticed to make a go of it?

Based on what I found so far in a multitude of searches, here are some of the missing pieces needed to probe and save the probe data to a file in Mach3:

Example of probing a rectangular area in Mach3, displaying and explaining the magic gcode generated from the Mach3 Art Fenerty Digitize plugin, from
http://www.cambam.co.uk/forum/index.php ... 376.0;wap2
French original:
http://www.metabricoleur.com/t2083-nume ... ach3#28389
Note the gcode output from the digitizing wizard plugin screen shot jpg on page above
http://i47.servimg.com/u/f47/14/07/22/66/05_fil10.jpg

This is the gcode with ";" comments added by me:
(Digitize File)
M40 ;opens file for writing G31 data to, looks like it opens a dialog box and user must set .txt extension and type in the filename. I don't see an argument for a filename or file extension to be included in the this G40 command, so we rely on the user to do it. No big deal.
G92X0Y0Z0 ; zero x, y, and z coordinates to current machine position
F300 ; probe feedrate in mm/minute (looks a bit fast to me, maybe use 25mm/min here instead)
GOX0Y0Z4 ; move z axis up to safe height of 4mm
G31 Z-2 ; probe move z axis down -2mm or until contact is made; this is the actual first probing
G0Z4 ; move z axis up to 4mm above zero
G0X1Y0Z4 ; move x axis to next probe point, z axis still remaining at 4mm above zero
g31 Z-2 ;probe again on the second coordinate location
.... repeat a probing pattern here
M41 ;close the file. It now contains all the probe coordinates.

"The M40 calls a routine in Visual Basic (a. M1S) located in the folder macros Mach3, and contains just the "OpenDigFile ()" which has the effect to open a query file in which you will specify the file name under which the coordinates scanned will be saved. It is important to you then remember to put the extension. Txt at the end of the filename (Mach does not do automatically)"

"The second line G-code, G92X0Y0Z0 has the effect of moving all axes to 0 at the point where the sensor is located, so before starting the program, move your probe to the point that you consider the 0, 0, 0 in XYZ."

"The G31 triggers the descent of Z; when the entry of "digit" is activated by contact with the object, the coordinates are saved in text file."

"Here is an excerpt of this text file, with the X, Y and Z of each scanned point on a line.

0.00000,0.00000,0.00259
4.99917,0.00000,0.00259
10.00458,0.00000,0.00259
15.00375,0.00000,0.00259
20.00292,0.00000,0.00259"

Looks like simple comma separated ascii. Note that for this output to be clean, that is without unneeded axis letters, the user must go to config pull-down menu then select pins and ports,then mill options and UNCHECK add letters to coordinates. Also if you are not using the A axis then turn off 4th axis points. A slightly more advanced parser could check for letters first so as to make this step unnecessary.

Here is a bit more on the subroutine that gcode G40 triggers in Mach3, as taken from the Mach3 Programmers Manual:
OpenDigFile
Sub OpenDigFile()
This function opens a digitization log file. When OpenDigFile is executed a FileOpen dialog will be
displayed, and the user can select an existing file, or enter a new file name, to which digitization (probing)
data points will be written. Once opened, G31 commands will cause the probe trigger position to be
written to the digitization file. When digitization is complete, CloseDigFile() can be used to close the file.

gcode G41 triggers CloseDigFile()

Side note from the French website(same as referenced above) on how to convert the point data file from Mach3 into STL format for importing the point cloud output data into typical 3d graphics programs:
http://www.metabricoleur.com/t2083-nume ... ach3#28389
"I had to develop a small application in Visual Basic. Net to transform this "scatter" in a full 3D shape, and in a format readable by most of 3D softwares.This application, Mach3Digit2STL.exe is attached and the converted. Txt file created by Mach3 in a file. STL result is saved in the same folder and with the same name as the original text file, except that its file extension becomes."

So, there is a example of getting Mach3 to probe and save the point data to a simple text file. I have not tested it yet because I do not yet own a copy of Mach3, but I hope you find it useful to getting this part of things to work so my feature request can be implemented. Cheers.

This is fantastic, exactly what I need, thanks Doug.

I shall run some experiments with M40/M41 on my CNC machine, hopefully at the weekend.

P.S. I also dont have a registered copy of Mach3 either but have the demo version. I think the only difference is that there is a 500 gcode line limit on the demo version so this should be OK for my experimental needs.

I did a quick test of the M40/M41 codes in Mach 3 at the weekend. I can confirm it works in the demo version. Heres a snippet (numbers are in inches):
This was after about 7 probes and the board itself was sloping high to low, left to right so these increasingly negative Z's are expected.

I cant promise anything too quickly due to time but this might interest you nonetheless.

This is great! I can see the tilt in the numbers, so they look like real data, yes. I anxiously await your beta on this. When you have it, I am fully willing to put in some heavy testing time, because I actually need this feature to mill long y-axis paths on prototypes in acrylic after using a probe to determine the offsets with Autoleveler.

I am buying a probe from cnc4pc.com see:
http://www.cnc4pc.com/Store/osc/product ... cts_id=323
It is fairly priced at $135.00, and doubles as a tool height setting pad using a different insert.

So, I am excited to think I could Z probe using that probe device, Autolevel my gcode, and run my leveled gcode on any millable material, not just conductive materials. That should excite any cnc operator who does long cuts and suffers from un-levelness in the z aixs. I bet many look at Autoleveler, get excited, and then say, oh, it's just for conductive circuit boards, or Oh, I have to probe each time, or oh, there are not enough probe points for my long parts to insure accuracy. So I think this new feature is the right angle, but I am preaching to the choir I know.

Hopefully you can find the time to round trip the data through the files.

Let me review my latest brainstorm of the future Autoleveler process as I anticipate it:

1. The user clicks a button to select the gcode file.

2. The user can get dimensions and inches-mm scale from the gcode file, or enters the probing settings directly.

3. Click a button to generate a gcode file the original way, probe and mill as it works now. So far, everything works the same.

OR, click a Different button to generate a gcode file containing just the "probe and save probe data to a file", but no milling. If Linux cnc, set the filename of the probe data file, to be generated in the next step. If Mach3, pop an ok dialog up to remind the user to save it to a filename which is the same as their original g code project plus "ProbeData", and txt file extention for notepad convenience.

4. The user runs the probing gcode and a new file is generated by Linux CNC or Mach3 containing the numerical values for x, y, and z for each probe point. In LinuxCNC I think the gcode call can name it, in Mach3 I think we need to rely on the user to name it.

5. The user takes the probe data file from the cnc computer via dongle or file transfer, and opens it in Autoleveler by clicking a "open probe data file" button and browsing to the probe data file and selecting it.

6. The user can click a button to Create Pre-Leveled Gcode file using the probe data file, but the app should do a sanity check to warn the user if there is a mismatch in origin or dimensions of the two steps. Compare origins of probing and milling steps, and probed area vs milled area.
The pre-leveled gcode now gets generated using the same interpolation leveling functions as before, but now the results are saved in the output gcode file. Autoleveler has now done it's magic and put out pre-leveled gcode.

The user can now run the leveled gcode file on their cnc to mill their part, and they can stop and restart it without having to reboot or re-probe. If the user moves the part, they should re-probe and regenerate the leveled gcode, unless the fixtures are highly precise and repeatable, but I would probably always re-probe for PCBs because the tolerances are very tight and drilling into the waste-board will move things some between jobs anyway I imagine.

Here are some "would be nice if..." ideas:
1. Would be nice if the settings panel would have option (using simple division in your code) to specify an amount of probings desired for each axis, and have the application evenly space individual probings out within the limits for each axis respectively.

2. Would be nice (or wise) to allow a way to save a "project" to contain the settings and file paths associated with a probing job. Could drop the projects into a unique folder for each, or could zip up everything into a single file ball and spit it back out onto the screen and inner variables upon reopening. That method would guarantee a full snapshot of original, probe data, and leveled files.

3. In a future release, the probing gcode could be folded up into a gcode "program" with loops and subroutines to reduce file size. This would perhaps be more elegant from a file size standpoint, but maybe less useful for re-running a probe step? Maybe the rerun issue is moot: Can probing to a file be stopped, backed up a few steps and rerun without breaking the file? Does it write over the old data or does it make new lines? How about with the way you do it now using variables in the Linuxcnc or Mach3? Subroutines would reduce gcode file size. There's an experiment for the future. Hmmm.

4. Autoleveler should have a two-stage probe z movement, one that moves down fast to near the probe height, and one that slooooly moves down the remaining distance. So, one relatively fast z down speed for getting close, and one for the final movement to touch the probe. This would speed up probing without sacrificing accuracy. 3 levels of approach speed anyone? Acceleration-limited approach to a near stop at touching? Beyond me, but like that old book on the high shelf of the old dusty library, I know that knowledge is up there somewhere. I didn't do calculus much, but I know what it smells like.


5 A barebones graphical display of the probing patten vs the outline of the milling area would be nice, but probably a bit complicated, involving a full tour of a java graphics library. The cool points would flood out with a color height map display of how crooked the surface is, as in probekins. http://wiki.linuxcnc.org/cgi-bin/wiki.pl?ProbeKins Even cooler to see the probe cloud data on the screen with layers of all of the above individually clickable. Oh, I digress.

Thanks for your effort to experiment, looks very promising that it correctly revealed the z tilt in the coordinates. We probably have real data. Let me know when you have a beta to test and thanks.

Thanks for the extensive feedback. I think we are pretty much on the same wavelength as what you describe is pretty much what I was thinking.

I have a few projects and work to do first but I have some ideas on a general restructuring of the website, autoleveller and other software tools which should lead to more frequent updates such as this in the future.

I have a similar request for reading a log of leveled data. The latest version of GRBL 0.9d now has a G38.2 straight probe instrruction that returns the X,Y,X of the probe point. (I have tested this). Output looks like this
>>> G38.2 Z10
[Probe:0.000,0.000,5.031]
With this we can now use GRBL (which does not have variable functionality) to get the level information, but then this level data needs to be merged with the gcode to get final gcode. If Autoleveler can then just do the conversion without using the #xxxx variables, this will then work for all those GRBL and Shapeoko people out there using their machines for engraving or pcb routing.

Funny this should come up right now because I am currently working on 2 big changes.

1. Some big website changes and additions.

2. Coinciding with the website changes there will be a new version of the Autoleveller which will include this probe logging /persistence function.

Since GRBL does not recognize variables, I will try to produce an output file which does not use variables in the GCode. Thanks for pointing that out.

P.S. I hope (but can't promise) to work on the Autoleveller and other CNC software ideas full time shortly so improvements and additions can appear more regular.

Unless I am missing something, I don't see any way of using GRBL and leveling without using these separate operations.
1.In your levelor, you select and read in the target .nc file, you then write out a probing nc/grbl code (considering the xy extents),
2. Run the resulting gcode on your board, then copy the screen log from gcodesender, save to text file,
3. Open your java autolevelor program and select this saved text file from the log. Scrub the log files, apply Autoleverl and then output the merged gcode file.

From what I can tell, GRBL 0.9d also stores the last probe values in a parameter [Probe:0.000,0.000,-13.778] but don't know how to reference it. It's not very straightforward like your autoleveler works with EMC2 or Mach3.

I saw this post today looking for the same thing https://github.com/grbl/grbl/issues/413 Chamnit response is variables are currently not an option. From what I am understanding, GRBL will only report back where the probe stopped.

I did find a pcb-leveler (with visualizer) from mw-development.com which is to manually enter the probe differentials. From what I can tell, he extrapolates between points.

For GRBL, for my purposes, I think I would be happy to just enter the probe data or have a parser take it from the log file from the probe sequence. Joe