|
|
Post by Thirsty on Jun 28, 2021 20:34:46 GMT
Hello OBDeleven Forum members, I've created a little website for some OBD related tasks which may help you. If you face any bugs please let me know so i can fix them. You can also suggest new features if you want to. Link: obd.itsezz.deThe following features are currently implemented: - SVM XOR - Tool for fixing SVM fault codes (works with 4 digit hex and 5 digit decimal).
- Coding Compare - Tool that shows you the differences between two hex values.
- Hex 2 Binary - Tool that allows you to see what bits are set for a given hexadecimal value.
- PR Codes Lookup - Tool that allows you to lookup the descriptions of your PR codes.
- Backup Converter - Convert OBDeleven backups to a flat format (like VCDS does).
- API - Details for using the new JSON API.
Under development: - Better visualisation for hex 2 binary
Changelog can be found in the spoiler below. 10.10.2022: - Updated the Backup converter (worked on a lot of "under the hood" stuff, added checkbox for advanced identification data)
- Tweaked the site appearance (almost all information is now inside accordions)
- Changed the API response format
- Some minor backend changes
04.03.2022: - Added some more text and tweaked the site appearance
- Added Hex 2 Binary converter
- Added PR Codes Lookup Tool
- Added JSON API
02.03.2022: - Added a BETA version of the backup converter
- Reworked the site so that the server handles the tasks instead of the client
- Tweaked the site appearance
- Added URL Parameter support for SVM XOR and Coding Compare (information regarding this feature can be found on the website)
21.09.2021: - Site is now in darkmode <3
- Removed some unnecessary text
- Reworked the coding compare output table (Thanks to dv52 (Australia) for the inspiration)
You can show toggle between the full comparison and only the bytes that differ (switch on the top right of the table)
With the button "Copy BBCode to clipboard" the result table is parsed into BBCode format which allows you to easily add it to a forum post (below you can see an example of the parsed BBCode table)
| Hex value 1 | Hex value 2 | Byte No | HEX | BITS
7 6 5 4 3 2 1 0 | HEX | BITS
7 6 5 4 3 2 1 0 | 00 | 00 | 0 0 0 0 0 0 0 0 | 11 | 0 0 0 1 0 0 0 1 | 01 | 00 | 0 0 0 0 0 0 0 0 | 11 | 0 0 0 1 0 0 0 1 |
If the site is not working correctly for you reload the site with CTRL + F5. Regards, Thirsty |
|
|
|
Post by dv52 (Australia) on Jun 30, 2021 4:54:53 GMT
^^^^ Arrrhhh........very nice indeed- thanks for the handy-dandy tools!!! I note that the flat file converter is still a work-in-progress (when I tried pasting a small snippet from a back-up file into the box, but I got "invalid format error"). Are there any other "prerequisites" to the form of the chosen data - maybe like, it has to include the first line of the standard OBD11 back-up file and the pasted data must finish in a certain manner? Also, there are currently 3 x options in the list of available delimiters - but these don't include CSV. I assume that this quasi standard delimiter will be included in a later version
Again- thanks
Don
|
|
|
|
Post by Thirsty on Jun 30, 2021 10:16:16 GMT
Are there any other "prerequisites" to the form of the chosen data - maybe like, it has to include the first line of the standard OBD11 back-up file and the pasted data must finish in a certain manner? Yes indeed. Currently the input must contain the text "Backup". This will be removed in later versions. Also, there are currently 3 x options in the list of available delimiters - but these don't include CSV. I assume that this quasi standard delimiter will be included in a later version Adding new delimiters is not much work. What delimiters do you want me to add? |
|
Deleted
Deleted Member
Posts: 0
|
Post by Deleted on Jun 30, 2021 21:55:25 GMT
just ran this on an android extract and compared to my excel converter works fine.
very helful tool thanks for doing this
appreciate this is still under dev but would also like to be able to do a full backup convertion and support for the IOS Format.
Cheers
|
|
|
|
Post by dv52 (Australia) on Jun 30, 2021 23:57:31 GMT
Also, there are currently 3 x options in the list of available delimiters - but these don't include CSV. I assume that this quasi standard delimiter will be included in a later version Adding new delimiters is not much work. What delimiters do you want me to add? I know that users of higher function tools like MS Excel can select different delimiters when parsing data- but if the raw data file is .CSV format, MS Excel and MS Access will automatically import the file without needing further intervention - so Comma Separated Values format (i.e. the ordinary comma ",") - please
Again. damn nice work - thanks!!!
Don
|
|
|
|
Post by Thirsty on Jul 1, 2021 7:41:05 GMT
@testeronline Thanks for the feedback  I currently don't have a full backup to test. Can you provide me a full android and a full ios backup? dv52 (Australia)I'll change the delimiter input from a fixed select to a textbox. That way the user can enter whatever delimiter he/she wants. |
|
Deleted
Deleted Member
Posts: 0
|
Post by Deleted on Jul 1, 2021 8:30:45 GMT
I will dig out one of each and PM link.
Just a word of warning on the CSV format (and it may not be an issue but just in case) the Adaption names in OBD11 have , in them eg Timing deviatio after chain replacement, intake bank 1 (some have multiple , in them) so not sure what that does when inporting.
Thanks
|
|
|
|
Post by Thirsty on Sept 12, 2021 18:15:43 GMT
Update: I've reworked the website. The full changelog is in the first post. Let me know if it was worth the time  |
|
|
|
Post by dv52 (Australia) on Sept 13, 2021 6:24:25 GMT
Thirsty " The XOR calculator works OK (when I enter a 5 x digit decimal number that is the equivalent to a hex number, I get the same answer in both cases - if I make the the appropriate decimal-to-hex conversion on the answer).
Just a further, small suggestion- for some reason, the XOR answer seems to drop leading zeros in the case of hex numbers. This means that the XOR answer isn't necessarily four digit number. OK if you know to add the leading zero again, but probably better to keep leading zero (if possible)
Also, can you include "add to clipboard" button on XOR App for answer ?
Don
PS: what happened to the flat file converter app for back-up data? I still can't easily parse OBD11's silly formats
|
|
|
|
Post by Thirsty on Sept 13, 2021 7:57:52 GMT
Just a further, small suggestion- for some reason, the XOR answer seems to drop leading zeros in the case of hex numbers. Can you give me some example values where this behaviour happens? Also, can you include "add to clipboard" button on XOR App for answer Sure will add that what happened to the flat file converter app for back-up data? I'm still working on it but since it didn't work 100% correctly i decided to remove it for now. Once it works as it should i'll add it again. |
|
|
|
Post by dv52 (Australia) on Sept 13, 2021 21:14:24 GMT
^^^^ Thirsty : the XOR function will return a value "0" whenever any of the Bit positions in the two hexadecimal numbers have the same value.
So - given that the decryption key is C9D2, the most dramatic example of an XOR answer with leading zeros is an old-value number in which the first 3 x digits are the same as the decryption key (i.e. C9Dx). For example, if the old-value number = C9D1, your calculator returns the answer 3 (or more correctly, hex0003).
Please don't take my suggestion as a criticism - it's most definitely NOT intended that way. It's just the pedant in me and an education background in which my teachers insisted on adherence to strict math nomenclature!!!!  Plus, I'm fairly sure that the MIB/MMI units in VAG cars that use SVM XOR require entry of a 4 x digit number (even if the leading digits are zero)
Thanks again for the good work - an excellent facility for the forum community (I wish that I had your code-cutting skills)
Don
|
|
|
|
Post by Thirsty on Sept 14, 2021 6:16:19 GMT
dv52 (Australia) Thank you for the hint and explanation i'll adjust that. Please don't take my suggestion as a criticism For me it's valuable feedback and not any kind of criticism Hopefully i have the time and energy to continue with the backup converter in the near future.  Edit: I've added the leading zeros and the copy to clipboard button. dv52 (Australia)Let me know if you have any other suggestions
|
|
|
|
Post by dv52 (Australia) on Sept 15, 2021 22:23:47 GMT
^^^^ Thirsty : Perfect!!! - thanks again
Don
|
|
|
|
Post by scientist on Nov 19, 2021 13:33:03 GMT
Update: I've reworked the website. The full changelog is in the first post. Let me know if it was worth the time An import of CSV (comma separated variable) data will fail if the data values contain comma characters unless the string containing the comma is quoted, that is surrounded by double or single quote marks. So if your imports are failing you can modify the import file and put quote characters around the offending data values. Hope this might help someone. |
|
|
|
Post by Thirsty on Nov 19, 2021 14:42:16 GMT
|
|
|
|
Post by Thirsty on Mar 2, 2022 19:39:38 GMT
The website has been updated. The changelog can be found in the first post's spoiler. Would appreciate some feedback especially regarding the backup converter. ( dv52 (Australia)) |
|
|
|
Post by dv52 (Australia) on Mar 6, 2022 21:59:23 GMT
Thirsty - I do like your your long-code comparison tool - very nice indeed (well done)!! Alas, the flat file conversion tool doesn't work for me (again). Maybe the problem is one of expectation (i.e. my expectation). I'm not sure if you are familiar with the way that VCDS does a back-up. It's called an "adaptation channel map", or "admap" for short.
Here's an example of the flat file structure from a module with a low channel count (in this case an Engine module):
;SW:04E-906-016-G HW:04E-907-309-A --- Engine
;Component:1,4l R4 TSI H08 7615, Coding:01250032242405082000
;EV_ECM14TFS01104E906016G,002010,EV_ECM14TFS01104E906016G.rod
;Tuesday,02,September,2014,17:59:38:03521
;VCDS Version: Beta 14.8.0 Data version: 20140822
;VCID: 26431A2059EACC0A89-8073
Activation of interior auxil. heating,active
Activation of start-stop function,active
Deactivate production mode,00 00 00
Production mode,00 00 00
Protective function,Enabled
Roller test bench mode; functional,not activated
Notice the format (in this case .CSV). Anything in the flat file that's header information is preceded by ";" and the separator for actual data fields fields is "," and the ONLY time a line-feed happens is for the start of a new adaptation channel.
Importantly, notice how ALL sentence structure is absent from the admap; no instances of tabs, indents. or line-feeds between data fields within the one channel. And especially no instances of "-------" (which OBD11 back-up files use a lot). And also importantly, the ONLY time a line-feed happens is for a new adaptation channel (not like OBD11 back-up - which seems to use line-feeds regardless of whether it's a new channel, or a new data-field within a channel)
The admap format is an extremely simple format which very easily can be parsed into analysis tools like MS Access/Excel.
For newer modules that use UDS/ODX protocols like the hex09 module on MQB platform cars, the adaptation channel count can approach 2,000. OBD11 back-up files for these modules are monstrously huge because they contain lots of unnecessary sentence formatting stuff. Yes, they may look pretty to read, but they are very clumsy to parse - IMO, of course
I can't believe that the designers at OBD11 have structured their back-up files because they think that users will manually read this information (no one is going to read a such huge data files manually). So, structuring back-up files with sentence format is a waste-of-effort - IMO, of course!!
Don
|
|
|
|
Post by whataboutthis on Mar 7, 2022 4:00:24 GMT
Just to add the IOS and Android formats are different but dev don't see many users caring and they obviously don't have any Standards processes.
Further there are still bugs in the backup process so bad in bad out applies.
Credit is due to Thirsty for getting this working (although your requirement is different)
I think the changes you would like are
1) Add ; at start off all header rows
2) Remove the Advanced ID Data from the extract
3) Remove the Control unit and Adaptation text from each row
4) ignore all the --- values (this i suspect is obd11 trying to normalize the data)
is this what you expect
Adaptations
Tire pressure monitoring display
Warning thresholds, tire pressure monitoring system
2
Roller test bench mode
---
Manual activation, possible
would be converted to:
Tire pressure monitoring display,Warning thresholds, tire pressure monitoring system,2 -- this is a three tier value
Roller test bench mode,Manual activation, possible -- this is a 2 tier value
Question
where do you expect to see the long coding is that the coding item in the header row
|
|
|
|
Post by Thirsty on Mar 7, 2022 7:28:11 GMT
1) Add ; at start off all header rows 2) Remove the Advanced ID Data from the extract 3) Remove the Control unit and Adaptation text from each row 4) ignore all the --- values (this i suspect is obd11 trying to normalize the data) Can dv52 (Australia) confirm that these are all the changes? If yes that shouldn't be a problem to implement  where do you expect to see the long coding is that the coding item in the header row I think it is the one in the header. Imo the header can have a different format because it's a very small part in the big backups but i can try to make it look like the vcds backup header. If you guys have any other feedback regarding the website output feel free to let me know  Thank you very much for your input guys |
|
|
|
Post by dv52 (Australia) on Mar 7, 2022 23:07:38 GMT
Thirsty : yes, that is pretty-much it. The underlying design principle of the flat file format should be that (other than header information), the file should be as austere as possible - meaning that it should contain ONLY data - with each "Secondary" data-field separated by the chosen delimiter and line feeds should only happen for each new "Primary" data-field (i.e. in this case, with each new adaptation channel). There are a couple of other matters that we haven't discussed:
First: If you look at how adaptation channels are structured on OBD modules, you will find these 3 x data-fields: - Parent Name
- Child Name
- Setting value
This is the classic structure for a relational database. Problem is that sometimes the "Child" name is missing because the Parent channel doesn't have children. One of the issues with the converter is that the end flat file should have the same number of data-fields for EVERY data row. Meaning that it would be wrong for the flat file to allow both: - Parent Name,Child Name,Setting value
- Parent Name,Setting value
The converter MUST have the same number of data fields in each data row - or said another way, the same number of delimiters in each data row (i.e. the same number of commas in each data row for in a .CSV file).
The way that the other diagnostic device does this is by concatenating the Parent Name and the Child name (where one exists) with a "-", so that the row data structure is Parent Name-Child Name,Setting value
Second: Another issue is how the converter handles conflicts between the chosen data-field delimiter and the same delimiter when it appears as a sentence separator. This is always a tricky issue when creating flat files from sentence-rich databases. I'm not sure how familiar you are with this problem, but there's an example in whataboutthis post:
Adaptations
Tire pressure monitoring display
Warning thresholds, tire pressure monitoring system
2
Notice the sentence comma between the words "threshold" and "tire". This is clearly very different to a comma that would result from a data-field separator in a .CSV file. If your converter simply allowed this to appear in the flat file, it would be confused as a data-field separator in any subsequent parsing process,
The normal solution for this potential conflict is to never allow sentence commas in .CSV files- usually the fix substitutes these with another symbol. This normally involves the converter doing a first-pass of the data which substitutes ALL sentence-separators that are the same as the chosen data-field delimiter. This is then followed by the flat-file conversion.
whataboutthis question regarding the long-code string is a good one (yes, VCDS has it in the header part of admap file). Personally, I don't like the fact that VCDS buries the "coding" in with the rest of the header information - because it really isn't header data. Long-code is something separate and special (IMO). As a suggestion - maybe put it as the last item in the header information (preceded with the usual ";"). And most importantly - don't separate the Byte values with a space (in normal OBD11 fashion)
Again personally, I'm not fussed about the advanced information. I don't use this data much for diagnostics
Hope this helps and many thanks for listening to my suggestions (again) - and keep-up the excellent work!!
Don
|
|
|
|
Post by dv52 (Australia) on Mar 12, 2022 4:12:39 GMT
Thirsty : Not sure what you think, but how about adding a search facility for VAG error codes.
I'm not entirely convinced that it would be useful - but maybe a searchable table with data fields like this:
error code="Tiptronic switch, recognition-F189: open-circuit/short-to-positive" = 18164 (decimal) = 46F4(hex) = P1756 (OBD II)
Don
|
|
|
|
Post by Thirsty on Mar 12, 2022 11:19:26 GMT
adding a search facility for VAG error codes. I thought about that aswell but there are so many sites that already have that feature. First of all i'll prioritize working on the current features especially the backup converter. When all that stuff works as it should i'll maybe take a look at the error codes Thanks again for your time and input it's much appreciated |
|
|
|
Post by graaate on May 19, 2022 16:26:49 GMT
Thank u!Very useful |
|
|
|
Post by Thirsty on Oct 10, 2022 19:16:09 GMT
I finally had some time and motivation to work on the website again My main focus in this update was the backup converter. I tried to implement all the suggested changes. 1) Add ; at start off all header rows 2) Remove the Advanced ID Data from the extract 3) Remove the Control unit and Adaptation text from each row 4) ignore all the --- values (this i suspect is obd11 trying to normalize the data) 1) All header rows and non adaptations/advanced identification rows now have a leading delimiter. 2) I've added a checkbox were the user can decide if he wants to include or exclude advanced identification data. If the checkbox is not ticked the output will look like this | delimiter=; : ;Adaptations
Parent Name;Child Name;Setting value
Parent Name;Child Name;Setting value
...
;Advanced Identification
Parent Name;Child Name;Setting value
Parent Name;Child Name;Setting value
...If the checkbox is ticked the output will look like this | delimiter=; : ...more header stuff...
;Backup name: Engine
;Long coding: 091D00322464010B0000
Parent Name;Child Name;Setting value
Parent Name;Child Name;Setting value
...3) Done The structure is now Parent Name;Child Name;Setting value (where ; is the delimiter chosen by the user). 4) I have not removed these values from the backup Question where do you expect to see the long coding is that the coding item in the header row Yes if there's a long coding value available it is normally the last item in the header. Problem is that sometimes the "Child" name is missing because the Parent channel doesn't have children. One of the issues with the converter is that the end flat file should have the same number of data-fields for EVERY data row. Meaning that it would be wrong for the flat file to allow both: - Parent Name,Child Name,Setting value
- Parent Name,Setting value
The converter MUST have the same number of data fields in each data row - or said another way, the same number of delimiters in each data row (i.e. the same number of commas in each data row for in a .CSV file). I tested it with both an android and ios backup and as far as i can tell the structure for adaptations/advanced identification is always Parent Name,Child Name,Setting value. If there's no Setting value the structure looks like this: Parent Name,Child Name, which means that there are always two delimiters present. If there are several Setting values (seems to happen on ios backups only) the structure looks like this: Parent Name,Child Name,Setting value 1
Parent Name,Child Name,Setting value 2
Parent Name,Child Name,Setting value 3Second: Another issue is how the converter handles conflicts between the chosen data-field delimiter and the same delimiter when it appears as a sentence separator. This is always a tricky issue when creating flat files from sentence-rich databases. I'm not sure how familiar you are with this problem, but there's an example in whataboutthis post: Adaptations
Tire pressure monitoring display
Warning thresholds, tire pressure monitoring system
2 Notice the sentence comma between the words "threshold" and "tire". This is clearly very different to a comma that would result from a data-field separator in a .CSV file. If your converter simply allowed this to appear in the flat file, it would be confused as a data-field separator in any subsequent parsing process,
The normal solution for this potential conflict is to never allow sentence commas in .CSV files- usually the fix substitutes these with another symbol. This normally involves the converter doing a first-pass of the data which substitutes ALL sentence-separators that are the same as the chosen data-field delimiter. This is then followed by the flat-file conversion. The only sentence separator i've encountered in adaptations/advanced identification was the comma. The converter removes all commas in the backup to prevent problems. If it's necessary i can also remove the colon in the header rows but i think that this should be fine. I've also made some additional changes in the backup converter backend (fixed some special cases etc). I think it should perform pretty solid now Apart from that, I made some visual changes to the website with the aim of improving clarity and work flow. The other minor changes are listed in the changelog spoiler of the first post. Let me know what you guys think about the new changes and of course feel free to test all the new stuff out. Last but not least as always feedback is much appreciated Regards ThirstyDE |
|
|
|
Post by whataboutthis on Oct 10, 2022 21:11:05 GMT
Thanks very much for all your hard work.
Will give a whirl this week and provide feedback asap.
|
|
Plus, I'm fairly sure that the MIB/MMI units in VAG cars that use SVM XOR require entry of a 4 x digit number (even if the leading digits are zero)
