Datalogging and data analysis - How do you get the most out of your datalogs? - Ford Mustang Forums : Corral.net Mustang Forum
 1Likes
  • 1 Post By Jack Hidley
 
LinkBack Thread Tools Display Modes
post #1 of 24 Old 09-24-2018, 12:03 PM Thread Starter
Registered User
 
Trader Feedback: (11)
Join Date: Nov 1999
Location: Where the Navy sends me...
Posts: 4,144
Datalogging and data analysis - How do you get the most out of your datalogs?

I'd like to start a discussion of data analysis - how you take the datalogs from whatever engine management system you're using and turn all that data into information that you can use to correct / improve your tune. I'm hoping some of the experienced tuners (professional or amateur) can chime in with their methodology for analyzing datalogs, tips or tricks they use to help them pull the pertinent information out of the thousands and thousands of lines of data, etc.

I've had a Tweecer RT on my '89 for years, but have never really taught myself to do much actual tuning. I've used the datalogging feature on many occasions, but I don't feel like I'm able to effectively turn those datalogs into useful updates to the tune. I have EEC Analyzer, and I've used the functions there, but I haven't gotten much out of that - seems like the changes that it makes are pretty small, which would lead me to believe I'm not feeding it the right info / good enough data.

In a thread a month or so ago, someone commented about having 4-1/2 minutes of datalogging, and was told that wasn't enough for a tuner to do anything with. So, how much data is enough? I'm sure that varies depending on where you are in the tuning process (just starting vs working on fine tuning a "mature" tune). How do you go about determining how much data is "enough" for those different scenarios? And how does one ensure that you get the right kind of data as you're driving (hitting the right load or the rights part of the MAF curve or whatever)?

Once you have the datalog, how do you go about processing the data to pull out what you need? When I drive around and take a ~20min datalog with the TweecerRT, that results in about 14,000 lines of data. It's a lot to look at, and I kinda get lost.
- If I put the data into Excel, there are a variety of ways I can use Conditional Formatting to highlight data - turn all the OL lines one color, CL lines another color; if MAF is > some value, turn it a color, etc etc. But even if I do that, I'm still left with tens of thousands of lines in the spreadsheet, so it's not easy to see trends at a glance. So, if you're looking at a spreadsheet, what tools do you use to pull out the data you're trying to focus on?
- If I put the data into Excel and graph it, I end up with a massive graph (again, tens of thousands of data point), which makes it pretty much impossible to pull out small changes or short duration events. So, if you graph the data, how you do it in a manner that allows you to see what you're trying to find?

For those of you who are experienced and good at doing this, these are probably stupid questions. But I have a feeling that I'm not alone when it comes to figuring out how to effectively use datalogging to help me improve my tune, so I'm hoping that some discussion here can help us all make the most of whatever engine management we're using.


'89 GT convertible, N/A 357W, MM front susp w/425# coil-overs and Koni D/As, MM TA/PB rear setup (Koni Yellows, 390-430# springs), Stoptech 332mm / 12" Baers, MM 6 point rollbar, Maier 1.5" flared fenders & quarters, 18x9.5" front/18x10.5" rear Enkei RPF-1s (street), 18x10" CCW C-10s (track), etc...
Patrick Olsen is offline  
Sponsored Links
Advertisement
 
post #2 of 24 Old 09-24-2018, 12:18 PM
Registered User
 
CrucialProspect's Avatar
 
Trader Feedback: (0)
Join Date: Jan 2016
Posts: 916
I’m only pulling 7 or 8 pids. It’s useful to me to dissect test and tune night at the track. I can see wheel speed vs rpm and shift points to boost and make adjustments. I learned early that there are very few gauges you look at going down the track, and that’s how I use it. Sorry not much on the “tuning” side, but there are many facets of using the data collected.

CrucialProspect is offline  
post #3 of 24 Old 09-24-2018, 04:09 PM
Registered User
 
Trader Feedback: (9)
Join Date: May 2002
Location: New Braunfels TX
Posts: 4,477
I hate to say it but a comprehensive answer to that question requires a book, really. What you look at and how you look at it is pretty contextual to what you're trying to do.

I've found stuff by just dumping everything, looking at it all to find something that comes out and ask "why is this?".

Different specific symptoms often means looking at different things. What to look at is really more a knowledge of how all the components of the engine system work and interact than anything else.

Why is X happening? Well what are all the components that influence the behavior of X? Now let's pull all those things into our log and reproduce X while logging. Now we find parameters in the log that are not normal or in contradiction, and analyze further until we've determined root cause. Now adjust and/or fix the root cause, and re-check by logging again until we confirm everything is as expected both in vehicle behavior and nominal parameters.

'94 GT, White, Cobra hood & bumper cover, '95R wheels
393w, Vic Jr 60cc, XE282HR, Victor 5.8 EFI, 80# injectors, Hooker LT 1 3/4 to 3", Borla proXS
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

T56 w/ AJE xmember, McLeod street twin, MM/Bilstein '03 IRS, JRi Agent47 SLA
Megasquirt
seijirou is offline  
Sponsored Links
Advertisement
 
post #4 of 24 Old 09-24-2018, 08:37 PM
Registered User
 
indy2000's Avatar
 
Trader Feedback: (3)
Join Date: Oct 2001
Location: vancouver
Posts: 21,469
you need, a need to log

ie what is it that you need to address?

this will dictate what you need to log, and how much data you need.

89 GT gr-40, KB2200(10psi), tweecer R/T,LC-1, 22gal cell, w/tailpipes.
indy2000 is offline  
post #5 of 24 Old 09-24-2018, 10:33 PM
Registered User
 
CrucialProspect's Avatar
 
Trader Feedback: (0)
Join Date: Jan 2016
Posts: 916
Indy hit the nail on the head.
CrucialProspect is offline  
post #6 of 24 Old 09-24-2018, 11:10 PM
Registered User
 
midnightlaundry's Avatar
 
Trader Feedback: (3)
Join Date: May 2004
Location: Westland, MI.
Posts: 2,146
DL is great. You make changes and log. Then tweek some more. Set up your safeguards like AFR min/max, max boost.. so you don't blow it up as you go.. YOu can calculate horsepower by injector duty cycle.. Here's one I just did Saturday. I've noticed that traction is a factor the colder the weather gets. I need to get a sensor on a front wheel so I can get Traction Control going..

I have the cursor where the tires spin starts at 77mph in 3rd gear at 8psi.. It's scary if you're not prepared for the back end to come around at 80mph.. The rpm line should be linear, but it's easy to see that it spikes, then you can see where the TPS line drops when I let off the pedal and then get back in it..

https://c2.staticflickr.com/2/1931/4...0bb49f9e_o.png
midnightlaundry is offline  
post #7 of 24 Old 09-25-2018, 04:22 AM
Registered User
 
CrucialProspect's Avatar
 
Trader Feedback: (0)
Join Date: Jan 2016
Posts: 916
Essentially, that’s what I’m doing, by monitoring a few pids, I can notice different things such as wheel speed, rpm, boost etc. by noticing the slips in rpm and wheel speed, I can tell when I’ve lost traction and can back boost down at that point.

However, if this is aimed at actually tuning a vehicle, I wouldn’t dump all pids in a log, you won’t know where to start. You gotta know what you are planning to change, or a problem area.

If you want to know at what point your iat spikes for example, you can record that data on a drive and you’ll see obviously, when wheel speed is not moving, you can see how fast iat spikes, and by how much. This seems all common sense, a stationary vehicle may see increase on inlet temps stationary. But given the data value, you can see just how hot it gets, for how long, and how it recovers once you start moving again, and be able to make changes.
CrucialProspect is offline  
post #8 of 24 Old 09-27-2018, 03:45 PM Thread Starter
Registered User
 
Trader Feedback: (11)
Join Date: Nov 1999
Location: Where the Navy sends me...
Posts: 4,144
Quote:
Originally Posted by seijirou View Post
I hate to say it but a comprehensive answer to that question requires a book, really.
Well, I did pick up Greg Banish's book about tuning. Now I just need to read it, and see if it gives me the sort of pointers I'm looking for.

Quote:
Originally Posted by seijirou View Post
I've found stuff by just dumping everything, looking at it all to find something that comes out and ask "why is this?".

Different specific symptoms often means looking at different things. What to look at is really more a knowledge of how all the components of the engine system work and interact than anything else.

Why is X happening? Well what are all the components that influence the behavior of X? Now let's pull all those things into our log and reproduce X while logging. Now we find parameters in the log that are not normal or in contradiction, and analyze further until we've determined root cause. Now adjust and/or fix the root cause, and re-check by logging again until we confirm everything is as expected both in vehicle behavior and nominal parameters.
Thanks, that all makes good sense. I just struggle with the practical application of the methodology.


Quote:
Originally Posted by indy2000 View Post
you need, a need to log

ie what is it that you need to address?

this will dictate what you need to log, and how much data you need.
Ahhh, so once you know, you just know. Got it!

'89 GT convertible, N/A 357W, MM front susp w/425# coil-overs and Koni D/As, MM TA/PB rear setup (Koni Yellows, 390-430# springs), Stoptech 332mm / 12" Baers, MM 6 point rollbar, Maier 1.5" flared fenders & quarters, 18x9.5" front/18x10.5" rear Enkei RPF-1s (street), 18x10" CCW C-10s (track), etc...
Patrick Olsen is offline  
post #9 of 24 Old 09-27-2018, 04:12 PM
Registered User
 
Trader Feedback: (9)
Join Date: May 2002
Location: New Braunfels TX
Posts: 4,477
Quote:
Originally Posted by Patrick Olsen View Post
Thanks, that all makes good sense. I just struggle with the practical application of the methodology.
If you have some example scenarios perhaps we can walk through ideas on approaching the triage.

Part of it too is what types of data is available, often called 'channels' or 'PIDs'. Things like RPM, coolant temperature, etc.

Another thing to understand is that the data on a channel is a stream of samples, and those samples have a rate, often called resolution. (think high vs. low video frame rate)

Some systems have powerful enough processors to run the car and develop samples at high resolution for every available channel so you can literally do kitchen sink logging every time. My system is like this, so I can dump everything and see if anything jumps out even if I don't have a specific symptom that I'm aware of.

Other systems don't have that much CPU horsepower, so for them you need to consider and select the appropriate channels relevant to the symptoms. Too few channels and you don't get the right story, too many and the resolution of them all is poor which is in effect blurring or blending the data over time making it hard or impossible to see fine detail. For example if you have a sample rate of 10 per second, but have a problem where you lose RPM for 1/20th of a second it is possible for your channel to miss that moment. This is more the case with factory ECUs. This is also why many others replied the way they did, because this is the reality for them.

Back to my first point about what channels are available.. how you approach triage is going to be heavily influenced by this. If you could only pick 10, consider what channels you would find valuable for your first look to triage an engine that is rhythmically running rough on a 6cyl that has per-cylinder EGT probes vs. one that does not.

'94 GT, White, Cobra hood & bumper cover, '95R wheels
393w, Vic Jr 60cc, XE282HR, Victor 5.8 EFI, 80# injectors, Hooker LT 1 3/4 to 3", Borla proXS
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

T56 w/ AJE xmember, McLeod street twin, MM/Bilstein '03 IRS, JRi Agent47 SLA
Megasquirt
seijirou is offline  
post #10 of 24 Old 09-27-2018, 07:21 PM
Registered User
 
indy2000's Avatar
 
Trader Feedback: (3)
Join Date: Oct 2001
Location: vancouver
Posts: 21,469
Quote:
Originally Posted by Patrick Olsen View Post
Ahhh, so once you know, you just know. Got it!
So you have no reason to datalog, but you want to know what to datalog, oh i see!

Perhaps you can put some effort into it, and try to come up with a scenario?


89 GT gr-40, KB2200(10psi), tweecer R/T,LC-1, 22gal cell, w/tailpipes.
indy2000 is offline  
post #11 of 24 Old 09-29-2018, 10:45 PM Thread Starter
Registered User
 
Trader Feedback: (11)
Join Date: Nov 1999
Location: Where the Navy sends me...
Posts: 4,144
Quote:
Originally Posted by indy2000 View Post
So you have no reason to datalog, but you want to know what to datalog, oh i see!
Not sure where you think you see that, because that isn't what I'm asking at all. Thanks for trying to help, though.
Patrick Olsen is offline  
post #12 of 24 Old 09-30-2018, 07:17 AM
Corral Elite Member
 
Saleen414's Avatar
 
Trader Feedback: (2)
Join Date: Feb 2012
Location: Levant, Maine
Posts: 2,320
It helps to understand what pids are inputs and which ones are outputs. base example, TP, IAT, RPM, MPH, Coolant temp are all inputs. Some base outputs are Inj D/C, Ign timing, If you are using it, EGR function, boost control ect ect. Once you know these, then the rest can start to make a lot more sense.

1989 Saleen 414 358cid SBF 2.8L Kenne Bell S/C
26 X 8.5 Hoosier
Best 1/4 mile time 8.623 @ 159.82 mph with a 1.34 60' (5.57 1/8th at 128)
Best 1/8 mile time 5.339 @ 133.46 mph with a 1.29 60'
08' Bullitt #4097 N/A 340ish RWHP <<<<<FOR SALE!
Saleen414 is online now  
post #13 of 24 Old 09-30-2018, 10:20 AM
Registered User
 
indy2000's Avatar
 
Trader Feedback: (3)
Join Date: Oct 2001
Location: vancouver
Posts: 21,469
Quote:
Originally Posted by Patrick Olsen View Post
Not sure where you think you see that, because that isn't what I'm asking at all. Thanks for trying to help, though.
I repeat,

Perhaps you can put some effort into it, and try to come up with a scenario?

or do you want me to give you an example?

I just rectified a specific surge issue on mine.

89 GT gr-40, KB2200(10psi), tweecer R/T,LC-1, 22gal cell, w/tailpipes.
indy2000 is offline  
post #14 of 24 Old 11-06-2018, 04:06 PM Thread Starter
Registered User
 
Trader Feedback: (11)
Join Date: Nov 1999
Location: Where the Navy sends me...
Posts: 4,144
If using an example of something you've done makes it easier for you to explain, then feel free to do so. In my head, the answer(s) to what I'm trying to ask shouldn't really depend on what issue(s) one is working to resolve. I left it generic because I would think the answers will apply regardless of what platform one is working on or what datalogging tool(s) one is using. Data is data, and how one sorts the data is what I'm getting at.

'89 GT convertible, N/A 357W, MM front susp w/425# coil-overs and Koni D/As, MM TA/PB rear setup (Koni Yellows, 390-430# springs), Stoptech 332mm / 12" Baers, MM 6 point rollbar, Maier 1.5" flared fenders & quarters, 18x9.5" front/18x10.5" rear Enkei RPF-1s (street), 18x10" CCW C-10s (track), etc...
Patrick Olsen is offline  
post #15 of 24 Old 11-06-2018, 05:24 PM
Registered User
 
indy2000's Avatar
 
Trader Feedback: (3)
Join Date: Oct 2001
Location: vancouver
Posts: 21,469
iíve explained quite appropriately

Data is just data, without knowing system design, its quite useless data

if you dont understand, i can give you an example if it helps you understand?

grab a copy of the GUFB and you can follow along, you will see actual PIDs involved.

Knowing what to datalog, will require you to understand system design

89 GT gr-40, KB2200(10psi), tweecer R/T,LC-1, 22gal cell, w/tailpipes.
indy2000 is offline  
post #16 of 24 Old 11-22-2018, 04:05 PM
Registered User
 
Jack Hidley's Avatar
 
Trader Feedback: (5)
Join Date: Apr 2000
Location: Danville, CA USA
Posts: 3,041
By far the most useful technique is to graph two different logged channels on a scatter plot.

https://en.wikipedia.org/wiki/Scatter_plot

This allows you to see the actual relationship that exists between the two. In most cases, you will want to gate one or both channels. Gating means to remove the data when certain conditions are not met. For example, if you want to look at some parameters when the engine is in closed loop operation, you would gate the data with the open/closed loop flag. If you are working on idle control issues, you would gate the data so that any data with the TPS above a certain voltage is excluded.

A good example of using a scatter plot is this. Say you log both a/f ratio and engine load. Graph one versus the other, but only above a certain TPS voltage. What you probably want to happen is that the resulting data cloud looks like a curved line such that as the engine load goes up, the a/f ratio goes down. You can program Excel to do a curve fit to the cloud data, so you don't have to just look at it.
seijirou likes this.

Jack Hidley
Maximum Motorsports Tech Support
Jack Hidley is offline  
post #17 of 24 Old 12-04-2018, 03:15 PM Thread Starter
Registered User
 
Trader Feedback: (11)
Join Date: Nov 1999
Location: Where the Navy sends me...
Posts: 4,144
Quote:
Originally Posted by indy2000 View Post
iíve explained quite appropriately
Explained what quite appropriately? Which of your four replies in this thread explained anything at all?

Quote:
Originally Posted by indy2000 View Post
if you dont understand, i can give you an example if it helps you understand?
We already had this conversation once...
Quote:
Originally Posted by Patrick Olsen View Post
Quote:
Originally Posted by indy2000 View Post
or do you want me to give you an example?
If using an example of something you've done makes it easier for you to explain, then feel free to do so. In my head, the answer(s) to what I'm trying to ask shouldn't really depend on what issue(s) one is working to resolve. I left it generic because I would think the answers will apply regardless of what platform one is working on or what datalogging tool(s) one is using. Data is data, and how one sorts the data is what I'm getting at.

So yes, if you want to explain using an example (like the surge you just fixed), feel free to do so.

Quote:
Originally Posted by indy2000 View Post
Knowing what to datalog, will require you to understand system design
And once again, I'm not asking about what to datalog, I'm asking about interpreting the datalog so that the thousands of lines of data become useful information.



Quote:
Originally Posted by Jack Hidley View Post
By far the most useful technique is to graph two different logged channels on a scatter plot.

https://en.wikipedia.org/wiki/Scatter_plot

This allows you to see the actual relationship that exists between the two. In most cases, you will want to gate one or both channels. Gating means to remove the data when certain conditions are not met. For example, if you want to look at some parameters when the engine is in closed loop operation, you would gate the data with the open/closed loop flag. If you are working on idle control issues, you would gate the data so that any data with the TPS above a certain voltage is excluded.

A good example of using a scatter plot is this. Say you log both a/f ratio and engine load. Graph one versus the other, but only above a certain TPS voltage. What you probably want to happen is that the resulting data cloud looks like a curved line such that as the engine load goes up, the a/f ratio goes down. You can program Excel to do a curve fit to the cloud data, so you don't have to just look at it.
Thanks, Jack, that's exactly the sort of advice I'm looking for.

'89 GT convertible, N/A 357W, MM front susp w/425# coil-overs and Koni D/As, MM TA/PB rear setup (Koni Yellows, 390-430# springs), Stoptech 332mm / 12" Baers, MM 6 point rollbar, Maier 1.5" flared fenders & quarters, 18x9.5" front/18x10.5" rear Enkei RPF-1s (street), 18x10" CCW C-10s (track), etc...
Patrick Olsen is offline  
post #18 of 24 Old 12-04-2018, 07:14 PM
Registered User
 
indy2000's Avatar
 
Trader Feedback: (3)
Join Date: Oct 2001
Location: vancouver
Posts: 21,469
it was answered/explained in post #4

have you read the GUFB to understand all the PIDS and how the strategy works?

89 GT gr-40, KB2200(10psi), tweecer R/T,LC-1, 22gal cell, w/tailpipes.
indy2000 is offline  
post #19 of 24 Old 12-04-2018, 08:33 PM
Registered User
 
t32265's Avatar
 
Trader Feedback: (16)
Join Date: Oct 2012
Location: Casselberry Florida
Posts: 485
Garage
I datalog IAT pre and post intercooler, dome pressure, solenoid duty cycle, fuel pressure, oil pressure, water temp, trans temp, front wheel speed, rear wheel speed, boost via map, rpm, fuel used per pass, injector duty cycle, crank input, cam input, TPS, g force at launch, timing, AFR, EST VE, and baro pressure. these allow me to see what is going on with my car during a pass. If I saw injector duty cycle climbs to high then I now I am having fuel delivery issues. with out this data I might as well just have a carb. If this site would not have changed I could post a datalog you could see what is going on.
t32265 is offline  
post #20 of 24 Old 12-04-2018, 09:25 PM
Registered User
 
indy2000's Avatar
 
Trader Feedback: (3)
Join Date: Oct 2001
Location: vancouver
Posts: 21,469
but how do you interpret all those lines of data?

89 GT gr-40, KB2200(10psi), tweecer R/T,LC-1, 22gal cell, w/tailpipes.
indy2000 is offline  
post #21 of 24 Old 12-05-2018, 07:09 PM
Registered User
 
t32265's Avatar
 
Trader Feedback: (16)
Join Date: Oct 2012
Location: Casselberry Florida
Posts: 485
Garage
Quote:
Originally Posted by indy2000 View Post
but how do you interpret all those lines of data?
I have it set up with multiple pages and as you switch pages the timing of the datalog stays the same, I can go from sensors to timing to boost and so on.
t32265 is offline  
post #22 of 24 Old 12-06-2018, 04:19 PM
Registered User
 
indy2000's Avatar
 
Trader Feedback: (3)
Join Date: Oct 2001
Location: vancouver
Posts: 21,469
That is navigating, how about interpreting?

89 GT gr-40, KB2200(10psi), tweecer R/T,LC-1, 22gal cell, w/tailpipes.
indy2000 is offline  
post #23 of 24 Old 12-06-2018, 06:52 PM
Registered User
 
t32265's Avatar
 
Trader Feedback: (16)
Join Date: Oct 2012
Location: Casselberry Florida
Posts: 485
Garage
Quote:
Originally Posted by indy2000 View Post
That is navigating, how about interpreting?
it is pretty self explanatory, I am able to see what is going on at any point across the entire pass, if I see rpm spike I check rear wheel speed front vs rear to check for breaking traction if nothing is strange then I check engine rpm vs drive shaft speed. With the Holley ECU I clear up clutter by deselecting any given PID or set normal ranges and if something is out of range then I can pull a system log to find out what is going wrong to address it. Prior to the Holley I ran a blow through carb and thought I did not need data. I even tried using a GoPro to log my gauges since under WOT everything happens so fast that you don't have a chance to look down even on a closed course like a dragstrip. my car will hit 140+ mph in around 5 seconds, I barely have time to shift let alone monitor engine functions. If I added a knock sensor I could even datalog detonation but chose to read plugs instead since I have a large solid roller cam and the harmonics could cause a false signal. I an read fuel used at any point and get a rough estimate of RWHP from that alone. I know since tuning on the dyno by using HP numbers versus fuel consumed how much Fuel it takes to make horsepower on a scale till I reach peak efficiency of my turbos. Without having this information I would be racing in the stone ages.
t32265 is offline  
post #24 of 24 Old 12-18-2018, 08:29 AM
Registered User
 
Trader Feedback: (4)
Join Date: Aug 2010
Location: casper wy.
Posts: 1,033
The info you are looking for can be found at efidynotuning or you can buy Don lasota's book on tuning and data logging. Both will give you info on what each parameter effects.

For example ECT will effect fueling. If your car starts well in 40 plus degree weather but not at -40 your fueling could be wrong. One would need to look at this parameter and do the math per the multiplier then log. See where this is going. Only log the parameter's you need to find the problem. Learning to tune will help. Read the above suggested info. and you will understand.
wywindsor is offline  
Sponsored Links
Advertisement
 
Reply

Bookmarks

Quick Reply
Message:
Options

Register Now



In order to be able to post messages on the Ford Mustang Forums : Corral.net Mustang Forum forums, you must first register.
Please enter your desired user name, your email address and other required details in the form below.

User Name:
Password
Please enter a password for your user account. Note that passwords are case-sensitive.

Password:


Confirm Password:
Email Address
Please enter a valid email address for yourself.

Email Address:
OR

Log-in










Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page
Display Modes
Linear Mode Linear Mode



Similar Threads
Thread Thread Starter Forum Replies Last Post
For Sale Autometer Data Logger, Innovate Wideband, Spaghetti Menders A-DAWG Electronics and Entertainment 9 10-08-2011 11:48 AM

Posting Rules  
You may post new threads
You may post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On

 
For the best viewing experience please update your browser to Google Chrome