JEEP HACKING CAN-C / CAN-IHS / UDS ! (Reverse Engineering)

[jmccorm]

The Python code actually functions, but I've only got a single headlight at the moment, so it flashes just a single headlight no matter which one is called up (and it's normal pattern is temporarily overridden).

Both RGB headlight halos are now fully controlled by the Raspberry Pi. I've updated the Python code in this earlier message with the latest code which now sends independent patterns to both headlights. It also correctly reflects the vehicle's turn signal and hazard status on the correct headlight(s).

This stuff is getting good!

Sponsored

 

[redracer]

Awesome job!

I took a look at your two scripts and i see what you're up to... But, i have to wonder, why not just do the whole thing in Python? I expanded upon your snippet and tossed this together. it's not complete of course...

#!/usr/bin/python3

import can

bus = can.interface.Bus(channel='can1', bustype='socketcan',filter=[{"can_id": 0x291, "can_mask": 0xFFF}])

try:
  for msg in bus:
    if msg.data[2] == 0x04:
      for i in range (0,LED_COUNT_X,1):
        stripL.setPixelColorRGB(i,255,0,128)
    if msg.data[2] == 0x08:
      for i in range (0,LED_COUNT_X,1):
        stripR.setPixelColorRGB(i,255,0,128)
    if msg.data[2] == 0x0C:
      for i in range (0,LED_COUNT_X,1):
            stripL.setPixelColorRGB(i,255,0,128)
            stripR.setPixelColorRGB(i,255,0,128)

except:
  bus.shutdown()
  curses.nocbreak()
  stdscr.keypad(0)
  curses.echo()
  curses.endwin()
  raise

 

[jmccorm]

I took a look at your two scripts and i see what you're up to... But, i have to wonder, why not just do the whole thing in Python? I expanded upon your snippet and tossed this together. it's not complete of course...

Originally, it was a combination of things:

The Raspberry Pi that I'm testing and creating new RGB halo patterns with aren't actually in the vehicle. They're in the house along with two delicately wired RGB halos. That means there's no actual vehicle and there's no real CAN bus traffic to trigger anything.

I probably could work around this by figuring out the "vcan" interface which creates a virtual CAN network for these kinds of situations (and then using "canplayer" to replay log files, or I could just transmit similar messages onto the vcan interface as needed). And then I need to switch interface names when the code moves back-and-forth between the vehicle and the desk. It just seemed a bit troublesome. Creating files as status flags was easy.

Second, my Bash skills are much stronger, but my Python skills are exceptionally weak. As in, I didn't know the language at all and I just started programming with it two days ago. I'm currently in the monkey-see-monkey-do phase where I can emulate something I see elsewhere and then slowly modify it into what I actually want to do. But you're kind of taking that excuse away since you've given me a starting point to merge into my code. For shame! ?

As I set out to do this, I had concerns about the CPU utilization. I saw that your Python code was pegging a CPU at 100% and I wanted to avoid that. By contrast, the operating system our Bash shell scripts to sleep when there's no data to read and then wakes them up when there's something new on the CAN bus, so it's super efficient. But now that I'm playing with these RGB halos, I'm seeing that they're going to drive a CPU's utilization high anyways, so I suppose this concern no longer exists.

Finally, I preferred to use multi-threaded code where one CPU thread monitors the CAN bus and keeps track of the turn signal status and the other thread creates patterns and then streams them as a data pattern onto the halo lights. I didn't want one function to slow the other down. Bash scripts do a great job running independent of each other. That's likely to be way too difficult to do for someone who's just starting out with Python.

Now...

Most of these reasons have gone away now that I've dug further into it. Based on what I'm seeing, in Python I can still check the CAN bus without any real impact on the rate that the headlights are updated. So this reason has also gone away now that I've dabbled a bit further into this.

Okay, okay. You've twisted my arm! After a little more playing around here, I'll go ahead and convert this code to Python. ?

Running the two pieces independently and then using the filesystem for signaling between them works, but it's a poor solution for something that could be designed more cleanly. Running the RGB halo code in Python really does make more sense. I'll start heading that direction.

PS: Thanks for the Python code for me to get started with! A quick note, though. That's only going to fill a halo with a solid amber light the moment the data is refreshed on the CAN bus. The next time an update is sent to the RGB halos, a new CAN signal won't be there, so they'll revert back to normal.

The secret is going to be to set a persistent global variable with the last received turn signal status. Then the code would draw the turn signals based upon the variable, not based upon a new signal being received or not. Hope that makes sense?

 

[redracer]

As I set out to do this, I had concerns about the CPU utilization. I saw that your Python code was pegging a CPU at 100% and I wanted to avoid that. By contrast, the operating system our Bash shell scripts to sleep when there's no data to read and then wakes them up when there's something new on the CAN bus, so it's super efficient. But now that I'm playing with these RGB halos, I'm seeing that they're going to drive a CPU's utilization high anyways, so I suppose this concern no longer exists.

Yes, I worry about CPU load as well. This is why I'm calling the filter option when starting the can library, as that should allow the kernel to handle the filtering much more efficiently than the python library.

Finally, I preferred to use multi-threaded code where one CPU thread monitors the CAN bus and keeps track of the turn signal status and the other thread creates patterns and then streams them as a data pattern onto the halo lights. I didn't want one function to slow the other down. Bash scripts do a great job running independent of each other. That's likely to be way too difficult to do for someone who's just starting out with Python.

That's also something that I need to work on. To work with tkinter, i need to get away from single threading. From the python can library, it looks really easy to do, using the notifier function to call other sections of code when the appropriate ID packet has arrived.
https://python-can.readthedocs.io/en/master/api.html#notifier

 

[redracer]

Finally figured out the python can notify. Now I can call functions directly when a message arrives. this will work well with non-blocking code. Example below:

#!/usr/bin/python3

import can
import time

def newmsg(msg):
  if msg.arbitration_id == 0x2C2 and msg.channel == 'can0':
    print(str(msg.data[2] / 10))
    print(str(((msg.data[0]<<8) + msg.data[1]) / 100))
  if msg.arbitration_id == 0x322 and msg.channel == 'can0':
    rpmstr =  str(((msg.data[0]<<8) +  msg.data[1]))
    if rpmstr == "65535":
      rpmstr = str(0)
    print(rpmstr)
    print(str(((msg.data[2]<<8) + msg.data[3]) / 200))

bus = can.interface.Bus('', bustype='socketcan',filter=[{"can_id": 0x2C2, "can_mask": 0xFFF},{"can_id": 0x322, "can_mask": 0xFFF}])
Notifier = can.Notifier(bus, [newmsg], loop=None)

try:
  while True:
    time.sleep(0.2)

except:
  bus.shutdown()
  raise

 

[jmccorm]

A Simple Commercial Product Worth Producing:
Remote Start Climate Restore​

I wanted to outline a concept for a commercial product that's likely to be popular with any number of Wrangler owners (and FCA owners in general). And I won't mind one bit if someone takes this idea and runs with it. I suspect any number of vehicle owners will be glad, too.

This is a basic and bulletproof version of my remote-start HVAC software I've been working on. The difference between what I'm working on and the device I propose is that instead of it enabling full-blast heating or full-blast cooling upon remote start, it does the next best thing: it simply turns the HVAC system on and allows it to resume with it's previous settings. It's simple, reliable, and it solves 95% of the same. problem. And it's something people will pay for.

Before I get into the details, there's one simple question I'd like to address. For such a simple and worthwhile commercial product, why hasn't anyone done this before?

I think I've found two answers to that question. First, it simple, and it's commercially viable, but the idea doesn't work unless you obey a very simple rule. It thwarted my own efforts for some time until I finally stumbled my way to success. Second, as it turns out, someone already has implemented this before. But it was one of several features in a smaller device that's almost exclusively paired with a larger and much costlier product.

After discovering the technique, I did some more researched and found that this was first implemented in the PAC-audio BCI-CH41 head unit adapter that's commonly paired with LinksWell and Stinger head units. You can find it on Amazon, but it's not much of a direct consumer product.

They call it a "Remote Start Climate Restore" feature, and you can read a bit more about it from the product manual. This is integrated into a complicated wiring harness that's used to integrate third party head units with FCA vehicles. That's too much for most people, and this feature is enough to command it's own price as a standalone feature.

So how do you get the HVAC system to turn on after the vehicle has started? It's easy. You could either send an HVAC on/off toggle button signal or an HVAC fan speed up signal to the CAN bus. After the vehicle's been remote started, that's enough to turn on the HVAC system.... almost. But experimenting along those lines will typically result in failure. Why? There's one more thing you have to know, and it's plain simple.

You've got to wait. It isn't enough to send one of those CAN bus messages after the vehicle has been remotely started, but you have to wait for a while before sending the message or the input is not accepted. How long? Experimentation has put it around approximately 60 seconds on my own vehicle. But you don't necessarily have to set against a hard timer.

You could look at some of the HVAC status messages on a regular basis. If the HVAC system is offline, you'd send one of the wakeup keypresses. Wait for a period of time (10-20 seconds) and check the status again. If it's still not awake, resend the command until the HVAC systems has turned on or your attempts eventually time out.

The only wrinkle would be in detecting and handling an auto-defrost of the windshield when it gets below 40F in winter, but I'd leave that in your own hands to determine how you'd handle that edge case. All-in-all, this could be a fairly simply product that's easy to support and provides a great deal of customer value.

I'll leave this in anyone's hands to figure out if this is worth it to them to develop, produce, sell, and support.

PS: I'll still be giving away my own Raspberry Pi implementation of this, but that's likely far too troublesome for most people to meddle with.

 

[jmccorm]

I went ahead and purchased this item:

Zero2Go Omini – Multi-Channel Power Supply for Raspberry Pi

Just yet, I'm nowhere near suggesting that we try to standardize around this. But I'd like to see if this module does everything it claims it can do for us.

I'll be trying to put these features to work:

• three separate voltage inputs [battery, AUX 3, AUX 4]
• application-level voltage monitoring scripts for us to do our own decision making based on any of the three line voltages
• monitoring of AUX 4 for my own project which will launch or terminate RGB LEDs controlling software
• buck/boost converter as a regulated power supply that supplies the Raspberry Pi and it's accessories with 5VDC
• automatic shutdown of applications and OS, followed by a full power cut to the Raspberry Pi after AUX 3 loses power (either manually or automatically or when vehicle is powered off)
• While power is cut to the Raspberry Pi, the device continues to monitor for wake-up conditions using a much lower standby current (0.36mA @12VDC)
• full power restoration and OS boot of the Raspberry Pi after our primary wake-up condition is met (AUX 3 power on)
• full power restoration to the Raspberry Pi when GPIO-4 goes LOW (I will need to develop an external circuit that uses this to perform a wake-on-CAN functionality)

I decided to bite the bullet and I purchased a very tall metal Raspberry Pi case (that requires an external USB adapter to get WiFi to work) and then a (very much overpriced) 2GB Raspberry Pi 4B to replace what's currently installed in the vehicle.

So things will slow down for me just a bit until I get these new hardware pieces integrated back into the vehicle (along with the rest of the electronics for driving the RGB halos).

 

[Drdyer9051]

I'm ALSO IN THE "currently in the monkey-see-monkey-do phase where I can emulate something I see elsewhere and then slowly modify it into what I actually want to do."
can some one explain the Mask, when and what to use 0FFFF vs FFFF vs .....etc
thanks
Bradley

 

[jmccorm]

I'm ALSO IN THE "currently in the monkey-see-monkey-do phase where I can emulate something I see elsewhere and then slowly modify it into what I actually want to do."
can some one explain the Mask, when and what to use 0FFFF vs FFFF vs .....etc

I'm happy to help, but I need a bit more context. Can you provide a link, a reference, or some specific example of what you're looking at or trying to do? (The more you can give us some background information, the better we can help you.)

 

[Drdyer9051]

candump help file .....


"Examples:
candump -c -c -ta can0,123:7FF,400:700,#000000FF can2,400~7F0 can3 can8

candump -l any,0~0,#FFFFFFFF
(log only error frames but no(!) data frames)
candump -l any,0:0,#FFFFFFFF
(log error frames and also all data frames)
candump vcan2,12345678:DFFFFFFF
(match only for extended CAN ID 12345678)
candump vcan2,123:7FF
(matches CAN ID 123 - including EFF and RTR frames)
candump vcan2,123:C00007FF
(matches CAN ID 123 - only SFF and non-RTR frames)"


why the FFFFFFFF, DFFFFFFF, 7FF
if you want to read a specific ID what mask do you put... say in some of your examples you want to monitor second byte of 8 returned


edit:

or redracers python example this page


bus = can.interface.Bus('', bustype='socketcan',filter=[{"can_id": 0x2C2, "can_mask": 0xFFF},{"can_id": 0x322, "can_mask": 0xFFF}])


why FFF and not FFFFFFFF or 7FF or C00007FF

thanks
Bradley

 

[jmccorm]

why the FFFFFFFF, DFFFFFFF, 7FF
if you want to read a specific ID what mask do you put... say in some of your examples you want to monitor second byte of 8 returned

why FFF and not FFFFFFFF or 7FF or C00007FF

WOW. You may not have realized this, but you've asked a fairly advanced question. To fully understand the answer requires a complete understanding of binary and hexadecimal representations and how to convert back-and-forth between the two formats.

I don't know how much of that you'll know, so let me start by giving you a basic answer and then I'll start to get more complex from there. This won't be a quick answer, but hopefully it'll be a basic one.

Short answer:
The CAN mask is not used for filtering out bytes from the content of a message. Always use a CAN mask of 0x7FF or 0xFFF (they both act the same) unless you're doing something that's more advanced.

Long answer:
As defined in our basic CAN bus standard, a message ID will always be a number that's 11 bits long. What does that mean? In binary form, the CAN ID is going to be any combination of binary digits from 0b00000000000 to 0b11111111111.

If all eleven digits are set to 0, that's going to be the smallest possible CAN ID number. In hexadecimal, we represent that as 0x000. If all eleven digits are set to 1, that's going to be the largest possible CAN ID number. In hexadecimal we represent that as 0x7FF. So that's where our CAN ID numbers come from.

Now, you ask about the connection between the CAN mask and how you'd monitor the second byte of a matching message. The answer is that you don't, because those two concepts are not related. The CAN mask is used for a different purpose.

The CAN mask is used in association with the CAN ID to specify certain combinations of message IDs that you want to receive (and not to specify anything about the content of those messages).

Inside the CAN mask, when a bit is set to "1", that tells the operating system that the bit you specified in the CAN ID must be a perfect match in order for you to see the message. This is complicated, but hopefully a few examples will at least make it a little bit clearer.

If you specify CAN ID 0x322 and a CAN mask of 0xFFF, your tell the operating system that for a message to be a match, all the bits in a matching message's CAN ID will have to be the same as the ID you specified, which is 0x322. So an ID of 0x322 and a mask of 0xFFF will only give you messages where the CAN ID is exactly 0x322. But what if you use a different CAN mask?

Once again, let's say we specify a CAN ID of 0x322, but this time we provide a CAN mask of 0x00F. What messages will our filter (the combination of a CAN ID and a CAN mask) allow?

For the first two hexadecimal digits in our mask, we have a 0. This means, "we don't care, anything in these first two digits will be a match". In our CAN mask, the last digit is 0xF, which is binary, is 0b11111111. That's all 1's. What we're saying is that the last hexadecimal number must be an exact match.

So if we're accepting message IDs where the first two hexadecimal digits can be anything and the last hexadecimal digit must be a "2", then we're telling it to give us any CAN messages with a CAN ID that ends with the number 2.

A skilled programmer who is well-versed with binary and hexadecimal numbers can use this to create filters that specify unique and interesting message ranges. Like a CAN ID of 0x400 and a CAN mask of 0x400 should (unless I've made a mistake) specify that you want to see any messages with a CAN ID of 0x400 to 0xFFF. Of course, the highest possible message is actually going to be 0x7FF, so that's as high as we'll get.

Sometimes you'll see us use a CAN mask of 0xFFF instead of 0x7FF. If you don't fully understand how the bitmask works, just know that those two masks end up being completely interchangeable for our purposes.

Now...

If you've got a follow-up question about the CAN mask, go ahead and ask. If not, we can get to what probably is your real question, and that is, you want to know how to isolate and use a small section of a CAN message that matches your given ID and mask. Right?

If so, to answer that question, it depends quite strongly on what language you're using and (in some languages) how you're bringing in the data. So if you're interested, specify the language and we'll continue from there.

If the language is Python, that's fine, but I may have to volunteer @redracer for a response. ?

 

[jmccorm]

If the language is Python, that's fine, but I may have to volunteer @redracer for a response. ?

I'll go ahead and take a swing at providing an answer in Python. For our example, I'm going to use redracer's pycan.py source code that's in our repository.

In this section of code, he looks for message ID 0x277, examines the second byte, and compares it to see if it has a value of 2 (or "0x02" in hexadecimal). If it does, he then sets the string mtstatus to equal "N":

if msg.arbitration_id == 0x277 and msg.channel == 'can1':
      if msg.data[1] == 0x02:
        mtstatus = 'N'

What you have to know is that when you're specifying which byte you want to look at, Python does not begin counting with the number 1. Instead, it begins counting with the number 0. So the first byte would be at index 0, and the second byte would be at index 1.

Knowing this, you'd extract just the second byte of a response by using:

variable msg.data[1]​

I hope this did a fair job of answering the question and representing his work. But if I didn't quite cover everything you were looking for or you have additional questions, please, ask away!

 

[redracer]

In the simplest method that I can think of.... a Mask is a comparison between two sets of binary bytes. for example:
0xC9 in Hex is:

11001001 binary

if we mask that with 0xFF then we get:

11001001 = 0xC9
11111111 = 0xFF

results in
11001001 = 0xC9

but, what if we don't care about the first bit, otherwise knows as the MSB (Most Significant Bit),
then we want to mask against:

11001001 = 0xC9
01111111 = 0x7F

results in

01001001 = 0x49

Why would we want to do this? here is one example...

The brake pressure word is mixed with a moving or stopped status bit. The moving bit is 0x4000 . How do we separate them out? With a mask.

((msg.data[0]<<8) + msg.data[1]) & 0x0FFF)

Here I am combining two 8 bit words to make one 16 bit word, then masking out the moving status from the brake pressure using 0x0FFF.

 

[jmccorm]

In the simplest method that I can think of.... a Mask is a comparison between two sets of binary bytes.

To explain how redracer's answer is different from my own:

The part that I explained (and what you see in the man pages for candump) are for masks that are used to select which message IDs you want to receive. And for those, you'll generally want to provide the message ID you're looking for, and then use a mask of 0xFFF or 0x7FF.

We also make use of message ID masks in Python with lines of code that look like this:

bus = can.interface.Bus('', bustype='socketcan',filter=[{"can_id": 0x2C2, "can_mask": 0xFFF},{"can_id": 0x322, "can_mask": 0xFFF}])

You had asked another question about message ID masks, which was this:

why FFF and not FFFFFFFF or 7FF or C00007FF​

The answer is that 7FF and FFF would be used on a CAN bus that has 11-bit message IDs, like we have on the Wrangler. There's also an extended variant of the CAN protocol (not used here) which has 29 bits, and those would have a CAN ID mask like FFFFFFF.

Additionally, CAN ID masks can also be used to specify what CAN error messages you do and don't want to see. When that's being done, even with 11-bit CAN IDs, you're going to see a very long CAN ID mask like C00007FF being used, but that's some super-advanced stuff.

Meanwhile, when working with message data (the payload of a message and not the message ID number), you have the option of using a mask to manipulate the message data to select which bits (or bytes) you want to keep and which ones you don't. Any bits that you don't want to keep will be replaced (in binary) with 0s. Those work like redracer explained (and I'm glad he did because I totally missed that part of the answer).

So I'm hoping this better differentiates our two responses and explains why they're different, yet both correct. This can be confusing, so if you have follow-up questions, you're welcome to ask.

 

[Drdyer9051]

thank you both, that clears up a lot. would have taken a whole lot longer to decode on my own. I know just enough to muddle through to achieve what I want to do. programming is not my trade, just a hobby.

what I have seen lately on Vcans, easy enough for this monkey to do it....

sudo ip link add dev vcan0 type vcan
sudo ip link set up vcan0

for those that do not have access to jeep canBus yet. I believe that you can still learn from this discussion by taking a .log file as in page 3 of this thread, run it on a canplayer and learn to candump, cansniff, canplayer...etc then dive into python. It would be very hard without the help of members of this forum. Thanks

Sponsored