Analyzing the Attack Surface of Ivanti's DSM
Ivanti’s Desktop & Server Management (DSM) product is an old acquaintance that we have encountered in numerous red team and internal assessments. The main purpose of the product is the centralized distribution of software packages. The solution is comprised of a number of components including a pre-boot and remote support module. In this blog post, we will focus on its core functionality: software management on Windows hosts.
Ownership of the product has changed multiple times, and former holders include Heat, Frontrange, Enteo and NetInstall. There is a similarly long list of reported vulnerabilities including:
- CVE-2024-29821 - local privilege escalation
- CVE-2023-28129 - local privilege escalation
- CVE-2022-????? - remote code execution (silently patched 🤫)
Despite this, not much is publicly known about its architecture or security concepts. In January 2024, Ivanti announced that DSM will reach End-Of-Life in December 2026. We would like to use this announcement to shed some light on common security pitfalls and architecture decisions one should be aware of when using DSM for the remainder of its lifetime. By doing so, we hope to help other red teams to achieve their mission goals, just as it has helped us over the years and simultaneously enable defenders to better understand the risk that a misconfigured / unpatched DSM installation poses.
We will begin with an overview of the architecture, explain how the DSM agent is installed and how it obtains its configuration. Next, we provide insights into the storage of credentials and attempts to protect them, specifically focusing on the usage of RPC and providing details for two vulnerabilities that can be triggered via RPC. Finally, we will demonstrate how software deployment works from the perspective of an admin and how this vector can be utilized for lateral movement.
This blog post will be accompanied by the release of internal tools. While we acknowledge the potential for misuse, we firmly believe that transparency and open access to security tools are essential in enabling defenders to evaluate risk exposure effectively. The underlying issues within DSM have been publicly known for over a decade, yet many environments remain exposed. It is crucial to recognize that threat actors have likely developed or acquired the capabilities described within the upcoming blog post, while defenders and ethical hackers have lacked accessible and verifiable means of evaluating risk exposure.
By releasing tooling, we aim to empower security teams to test systems properly, confirm if they are vulnerable, and take appropriate action. Even without providing the tooling, the level of detail in the blog post will be sufficient for attackers to easily replicate the attack. We will also provide mitigation and detection guidance that can be applied to harden existing DSM environments.
Architecture
At a high level, the DSM software distribution module consists of the following five components:
-
the agent
- running on all managed hosts
- consisting of a Windows service and helper utilities running in the desktop session of the currently logged-in user
-
one or multiple software depots
- that store:
- distributable software packages
- two configuration files (the so-called Infrastructure Configuration Databases)
- several binaries including the current installer for the agent
- that are either hosted via SMB (default share name:
DSM$
) or WebDAV
- that store:
-
one or multiple management points
- used to change the configuration, create new packages, add assignment rules, …
- that expose their interfaces via SOAP
-
one database
- a database scheme that contains tables for various configuration settings
-
the console
- a rich client application used by those responsible to administer the software
- that connects back to the Management Points and depots
Wait? How does the authentication for remote installation work?
One essential question that pops up when dealing with central software deployment solutions is, which account is used to roll out the software packages. According to best-practice, normal users should not possess local administrative permissions, meaning the installation needs to take place either with a local high-privileged account or an Active Directory account with the respective permissions. The latter would mean that a directory account with local administrative access to all managed systems exists, and performs sign-ins on a regular basis. This account therefore makes for a very interesting target for an attacker.
DSM offers two seperate modes to gain the required installation permissions. One could either choose to use the identity of the DSM service running on the client (SYSTEM) or an explicit credential pair configurable in the DSM console. Following the second, more interesting path, the question of how these credentials are stored and end up at the managed hosts comes to mind. To answer this we took a closer look on the agent.
The DSM Agent - Installation
As mentioned in the Architecture overview, the agent installer can be found on the software depots.
The latest installer is typically located at \\[DEPOT_SERVER]\[DEPOT_SHARE]\SSI\dsm.msi
). The only parameter the installer requires is the location of a valid software depot.
The agent and its installer create multiple log files that are useful for debugging. The log path can vary and is configurable in
the DSM console. When manually installing the agent, the path can be controlled by appending the following parameter to the MSI call: LOGFILEPATH="C:\Wherever\You\Want\It"
.
The DSM Agent - Program Flow
Using ProcessMonitor it was possible to get a rough overview of the program flow. One of the first things that happens is that the application connects to the given software depot and syncs the file NiCfgSrv.ncp. According to the vendors manual,
this file represents the so-called Infrastructure Configuration Database (ICDB) which is synced to the local folder
C:\Program Files (x86)\netinst\
.
The DSM Agent - Impersonation
Interestingly a log entry, that occurred after the synchronization, indicated that the agent now had knowledge about an AD account bound to DSM.
09:11:44.446 0 nwcmclnt.dll: Role based login succeeded with code-white.local\dsm-depot_access
Creating an LSASS dump while the client uses the account, revealed the clear-text password, as expected:
$ mimikatz.exe
.#####. mimikatz 2.2.0 (x64) #19041 Sep 19 2022 17:44:08
.## ^ ##. "A La Vie, A L'Amour" - (oe.eo)
## / \ ## /*** Benjamin DELPY `gentilkiwi` ( benjamin@gentilkiwi.com )
## \ / ## > https://blog.gentilkiwi.com/mimikatz
'## v ##' Vincent LE TOUX ( vincent.letoux@gmail.com )
'#####' > https://pingcastle.com / https://mysmartlogon.com ***/
mimikatz # sekurlsa::minidump c:\data\lsass.dmp
Switch to MINIDUMP : 'c:\data\lsass.dmp'
mimikatz # sekurlsa::logonPasswords full
Opening : 'c:\data\lsass.dmp' file for minidump...
Authentication Id : 0 ; 7653762 (00000000:0074c982)
Session : NewCredentials from 0
User Name : SYSTEM
Domain : NT-AUTORITÄT
Logon Server : (null)
Logon Time : 05.03.2025 18:07:20
SID : S-1-5-18
[...]
kerberos :
* Username : dsm-depot_access
* Domain : code-white.local
* Password : MuesseGebeImmerDie[...]
This was seen as a clear indication that the credentials are stored inside the configuration database.
The Infrastructure Configuration Databases
On a software depot two ICDBs can be found: NiCfgLcl.ncp and NiCfgSrv.ncp. According to Ivanti the first only stores configuration data required locally on the managed computers, while the second stores configuration data for the entire DSM environment.
A first look at the files reveals that the keys and values within are structured in an unknown binary format.
[root@cw]$ xxd NiCfgSrv.ncp | head
00000010: 0c00 4164 644f 6e43 6f6e 6669 6700 0000 ..AddOnConfig...
00000020: ffff 0500 0100 0000 1200 ffff 0600 0100 ................
00000030: 0108 0c00 4b6e 6f77 6e41 6464 4f6e 7300 ....KnownAddOns.
00000040: 0100 0000 0100 0000 0001 0000 ffff 0600 ................
[...]
Apart from illegible binary gibberish, the files contain many ASCII strings. These often include several account names of the target environment and can easily be identified by searching for the target domain name within the ICDB:
[root@cw]$ strings NiCfg*.ncp | grep -i 'code-white\\'
code-white\dsm-bls_authenticati
code-white\dsm-client_proxy
code-white\dsm-runtime_service
code-white\dsm-depot_access
code-white\dsm-distribution-ser
code-white\dsm-service_installa
code-white\dsm-read-privkey
code-white\dsm-osd_share
Taking a look at the assembler code of the client binaries, specifically icdb.dll, it became clear that the format is proprietary. In order to properly read the configuration file two options came to mind:
- invest a whole lot of time and energy to implement a parser for the format OR
- use the existing code to handle the heavy lifting
Luckily the vendor ships the conspicuously named binary NCPExport.exe
that can be found in the root directory of any depot.
According to its command line help, it can be used to convert the proprietary format into XML.
C:\Program Files (x86)\netinst\NCPExport.exe /?
NcpExport.exe is running ...
NcpExport [/xml:<filename>] [/full] [/ovo | /ovo:<filename>]
Just calling the binary with the specified parameters however, ends with a permission check:
C:\Program Files (x86)\netinst\NCPExport.exe /xml:NiCfgSrv.ncp /full
NcpExport.exe is running ...
Initialization of NetInstall finished. Log file name is NcpExport.log.
NcpExportAuthentication.ini not found.
Try to login to NetInstall using current user account.
Initialization failed. User has no permission to load the NCP!
Using a disassembler one can follow the trace from the error string to the function referencing it. According to the assembler code, the error message is thrown if NILO.dll:NiloLoginUser returns zero. Besides printing the error message, the respective code block also sets the return value of the current function to zero.
Changing the JNZ
instruction to JZ
reverses this logic such that no error is thrown and the function returns one,
if NiLoginUser fails. This is enough to bypass the first check as the parent function relied on the returned status code
and aborted execution if the function returns zero.
After patching the binary at six other locations to bypass the subsequent checks, triggering the export succeeded. The number of checks is highly dependent on the version targeted and might vary accordingly. The beginning of the XML document is outlined below.
<?xml version="1.0" encoding="UTF-8"?>
<NCP version="6.0">
<NCPObject name="code-white.local" type="ORG" ID="65535">
<Description value=""/>
<SECTION value="AddOnConfig">
<VARIABLE name="KnownAddOns" type="STRING" value="nilSplit.dll|niMW.dll|[...] "STATUS_SPECIFIED="YES"/>
[...]
Several tags holding credential information can usually be found in the XML export including
accounts for read-access to the depots, the software installation user (if configured) and also
credentials for accessing the management database. Our export contained all these tags, but they were filled with the
dummy value **HIDDEN**
.
Going back to the decompiler, it turns out that this substitution is performed by NCPExport.exe
itself.
The program does this by comparing each XML field to the string password
and setting the register EDI
to one for a match. The value is only read if EDI
is zero, otherwise, the actual password is replaced with the string **HIDDEN**
in the output:
By also patching this logic, it is possible to reveal the contents of the password tags stored in the ICDB:
<SECTION value="ServiceSettings">
<VARIABLE name="UseService" type="BOOL" value="YES"/>
<VARIABLE name="AutoAdminDetect" type="BOOL" value="YES"/>
<VARIABLE name="AccountName" type="STRING" value="code-white\dsm-depot_access" STATUS_SPECIFIED="YES"/>
<VARIABLE name="Password" type="STRING" value="K1MXSOOEE*SUOEM;FOOBRM/SOQRXFO;BMBFSOEMAFO;LEBFOAU" STATUS_SPECIFIED="YES" PASSWORD="YES"/>
The password displayed above is obviously not the plaintext password. DSM applies various encryption schemes onto
credentials that are stored within the ICDB. These encryption schemes are described in detail in the
Multiple Crypto Algorithms section. For now, it is sufficient to know that encrypted
passwords in DSM start with the pattern k<DIGIT>
. Using this knowledge, encrypted password strings can be retrieved
from a binary ICDB without converting it to the XML format:
[root@cw]$ strings NiCfg*.ncp | grep -i '^k' | sort -u
K1MXSOOEE*SUOEM;FOOBRM/SOQRXFO;BMBFSOEMAFO;LEBFOAU[...]
k7|d5BaG76fK0vzMZvpyLt+wp8v2GrdukwnqqwsIPvh1qaxuzGjHLodKH739MriX[...]
[...]
Decrypting the K-asswords
Protecting passwords by storing them only in an encrypted form is generally seen as good practice. Yet, at the end of the day the DSM agent needs to obtain the credentials in clear-text to perform its software maintenance and installation tasks.
Thus, all information required to decrypt, at minimum, the installation user, should reside on the endpoints themselves. Given this, the relevant logic should be found in one of the agent binaries. Grepping for the strings decrypt and encrypt in the installation directory quickly reduced the number of possible candidates:
[root@cw]$ grep -ril 'decrypt' | grep -i dll | xargs grep -il 'encrypt' | sort -u
AddOns/xniSPD.dll
DUNZIP32.DLL
errprov/icdbmsg.dll
esiAdminLib.dll
magntext/cmsext.dll
magntext/icdbext.dll
NiCfgPrv.dll
NIRT.dll
NISmtp.dll
SI.dll
SWMSClntLib.dll
Going through these, magntext/icdbext.dll looks most promising as one of its functions references a lot of seemingly relevant strings.
Setting a breakpoint at the beginning and end of this function shows that it retrieves the k-string as input and returns the decrypted value as clear-text:
It also reveals that the number following the k
is used in a switch-case statement. The options lead to
different sub-functions with varying error messages. This implies that multiple encryption algorithms exist, and that
the algorithm is chosen by the digit at position 2 of the password string.
Having discovered this, the plan to build a stand-alone decryptor was abandoned, as the complexity of implementing multiple different custom ciphers was considered prohibitive. Instead, the strategy of re-using existing code blocks was favored once again.
As the decryption function is contained inside a DLL and DLLs are made for code reuse, couldn’t we just call
the respective export? Well, unfortunately no, since the function is not directly exported. Tracing back, the function
address references always ended up in a function address table in the .rdata
segment. After some additional digging it became clear that this memory location is part of a Microsoft RPC call table (ncalrpc).
As we will cover later in this post, the DSM agent exposes several different RPC interfaces which, among other things,
allow calling the decryption routine.
For now, we want to stick to resources that are shipped by the vendor. Fortunately, the agent comes with an RPC client DLL
(icdbclnt.dll
). This library exports a method that calls the decryption routine via RPC.
Therefore, the path for manual decryption looks straight forward:
- create a new C project
- link the RPC client DLL
- call the exported decrypt function
- profit
And yes, that is basically it! If you wonder however why your decryptor never returns from the function call in the DLL and why your CPU spikes up to 100%, you might want to check for some anti-tamper protections ;). A little binary patching once again allowed us to bypass these protections.
C:\Program Files (x86)\netinst\> decrypt.exe K1MXSOOEE*SUOEM;FOOBRM/SOQRXFO;BMBFSOEMAFO;LEBFOAU
[*] Decrypted: MuesseGebeImmerDieB[...]
Multiple Crypto Algorithms
The final question that needs to be answered is, can one decrypt all password blobs regardless of the chosen algorithm? Well, that depends. To make it a bit more clear we have created the following table.
Scheme | Cipher | Needs individual key |
---|---|---|
k0 | KryptObfuscated | No |
k1 | KryptMoreObfuscated | No |
k2 | KryptConventional | No |
k3 | KryptEnhanced | No |
k4 | KryptEnhanced2 | yes, value from the ICDB |
k5 | KryptClient | yes, HKEY_USERS\S-1-5-18\SOFTWARE\NetSupport\NetInstall\ClientKey SDDL D:P(A;;GA;;;SY) -> Access only with local administrative permissions on a managed host |
k6 | KryptEnvironment | yes, \\dsm-depot\ni$\config\key\private.key , access only for a limited AD user list defined in the DSM Console |
k7 | KryptAesRSA | yes, \\dsm-depot\ni$\config\key\aesrsa.key , access only for a limited AD user list defined in the DSM Console |
k8 | ? | yes, value from the ICDB |
Based on this, the only password-strings that should be considered safe are k6
and k7
.
K5 is decryptable as soon as an attacker gets administrative access on a managed host.
But DSM Still Needs Those Creds1!!111!
Explicit usage of k6
and k7
is easy to recommend, but does DSM not need to
decrypt the credentials anyway? To answer this question, we need to take a look at the different
accounts that are used by the software. We will focus on the following three, which are usually contained
within the ICDB:
- Depot Access User - This account provides read-only access to the software depot servers. Within an Active Directory environment, access permissions to network shares can also be granted to computer accounts. Therefore, it is usually sufficient to use the local SYSTEM account for this purpose, which eliminates the need for storing the depot access user password within the ICDB.
- Software Installation User - This account is responsible for installing software and requires high permissions to all managed endpoints. Fortunately, the Windows operating system has a built in privileged account named SYSTEM that can be utilized for this task. Configuring the software installation to be performed as this user prevents storage of another AD credential pair within the ICDB.
- Database User - The ICDB usually also contains a connection string to connect to the
DSM database. We do not really understand why one puts this information in a database that
is replicated to all endpoints, but well, we have to deal with it. Since endpoints do not
require direct connections to the DSM database, the database password should always be
protected by the
k6
ork7
scheme.
Check your local environment
To harden / check your local DSM environment the following steps are suggested:
- Implement the Zero Account Model
- Store as few accounts as possible in the ICDB.
- You can omit the installation account and depot access account if packages are installed using SYSTEM and network access carried out via the computer account (System -> Network Service -> Computer Account).
- Check ICDBs for residual ciphertexts.
- Run the following command to extract credentials from the config and check for ciphertexts that are not of type
k6
ork7
.[root@cw]$ strings NiCfg*.ncp | grep -i '^k' -A 1 -B 1
- Due to the binary format, there is no guarantee that the account in the line before/after the k-value is actually the one bound to each ciphertext. It is a good indicator nevertheless. If in doubt, change the password for the account via the console and verify that the intended value changes in the config as well.
- Run the following command to extract credentials from the config and check for ciphertexts that are not of type
- Rotate passwords and ensure usage of secure encryption schemes.
- DSM should automatically use the secure encryption schemes when the password of an account that is not needed on the managed hosts is changed. Ensure that passwords are rotated and not just re-encrypted.
- Rotate the passwords of all previously used accounts, including accounts listed in config backup files (
NiCfg*.ba*
), as encrypted values might still be found in log files/ICDB backups on formerly managed clients.
- Verify ACLs and remove low-privileged users.
- Verify that the ACL for the
config
folder is properly set on ALL depots. The ACL should be automatically set by the software but might have been changed manually. - Remove any low-privileged users from the ACL list.
- Verify that the ACL for the
- Limit permissions and monitor accounts.
- Ensure that the permissions of remaining accounts are strictly limited to their respective tasks.
- Access to the DSM console should be treated as an equivalent to providing direct administrative privileges to all managed systems. Therefore, accounts with DSM console access should be explicitly protected and monitored.
- Add SIEM rules to monitor the remaining accounts for unusual activity.
Notably, previous publications by the company SySS (SYSS-2019-019, SYSS-2015 - BSIDES and SYSS-2014-007) already addressed the topic of decryptable password blobs in the DSM configuration databases. We have tried to complement the existing research and distinguish the K-blobs that are secure from those that are not.
DSM RPC Analysis
In this section, we will explore the different RPC interfaces that are exposed by the DSM agent component. We will showcase two vulnerabilities (remote code execution and local privilege escalation) that were exploitable via RPC and that we reported to Ivanti. Both vulnerabilities have already been patched with the latest having being remediated over 20 months ago. We will also take a look at how credentials from the DSM management database (ICDB) can be decrypted via RPC.
We have released tools related to this blog post on our GitHub Page which you can use to experiment with the RPC related techniques and vulnerabilities.
Foreword on Analyzing Microsoft RPC Interfaces
When it comes to analyzing RPC interfaces, RpcView has always
been an indispensable tool. At the time of writing, however, the x86
security callback enumeration in RpcView contained a bug, which is important in this case as DSM RPC interfaces are exposed through 32bit processes. We therefore decided to use rpv-web which has the added advantage of providing a Ghidra integration to speed up the analysis.
This article does not provide a detailed introduction about Microsoft’s RPC technology nor how to analyze it from a security perspective. If you are not yet familiar with the technology we recommend some prior reading like Offensive Windows IPC Internals 2 by @0xcsandker or An Overview of MS-RPC and Its Security Mechanisms by Ben Barnea. That being said, the rest of this article can probably be followed without deeper understanding of RPC.
RPC Interfaces of DSM
Running rpv-web
on a system with a DSM agent installed reveals that the process mgmtagnt.exe
acts as an RPC server.
The process exposes numerous RPC interfaces, whose code is mostly not located in the binary itself but in various DLLs
found in the agent installation folder.
The set of RPC interfaces exposed is determined by the startup parameters. When running without
any arguments, as is the case for the Windows Service Ivanti DSM Core Services
, the following interfaces are available:
UUID | DLL | Method Count | Name | Annotation |
---|---|---|---|---|
cd67fbf2-b9b5-4dcc-9ee5-8438cf925869 | mgmtagnt.exe | 6 | DSM Infrastructure Management Agent Core | Core Service Enumeration |
bf0517d4-7d10-46ad-b53e-f7743f76cafb | clntext.dll | 19 | DSM Client Core Extension | Enteo Client Interface |
38c90511-888b-4dd6-b6f6-09221a2d0386 | clntext.dll | 2 | DSM Client Core Extension | enteo MSI Privilege Elevation Interface |
9bdeb192-c051-44f4-a3e2-f8b730e88567 | clntext.dll | 6 | DSM Client Core Extension | Notification Interface |
5a622071-c10c-4160-8327-20e943f41644 | icdbext.dll | 20 | DSM ICDB mgmtagnt extension | |
4f11e644-a307-4d14-a235-5ce6e0a131c8 | icdbext.dll | 9 | DSM ICDB mgmtagnt extension | |
e47ff1a4-eb5c-41eb-9f16-d7c645345e29 | icdbext.dll | 2 | DSM ICDB mgmtagnt extension | |
6e603c2a-291c-4a0a-b6a9-59d96459ec3d | MICacheManager.dll | 2 | Management Information Cache Manager | MICacheManager interface |
ba0ad026-2f7d-49e4-9091-89cd7a650b31 | LocalJobManager.dll | 24 | Local Job Manager | LocalJobManager Interface |
64e6a1b9-1282-4ed7-b564-1edc86afac71 | cmsext.dll | 13 | DSM CMS extension | |
b8007262-7c16-492a-928b-d3689992ef05 | cmsext.dll | 14 | DSM CMS extension | |
78d9e903-698f-4beb-9f84-c7788ac04c18 | cmsext.dll | 1 | DSM CMS extension | |
5caa6658-60fa-44c4-aecf-98acaaf09682 | FpsCacheManager.dll | 30 | File Package Services Job Manager | FpsCacheManager Interface |
c5978785-d12d-40e9-a1c9-0c28faa4a944 | FpsCacheManager.dll | 5 | File Package Services Job Manager | FpsCacheManager Offline Media Interface |
fc331411-2f4b-4fb8-b677-e2a5d01f9885 | FpsCacheManager.dll | 2 | File Package Services Job Manager | FpsCacheManager Tools Interface |
b8ff9393-2cdf-427b-9bb8-441226b71ed6 | FpsCacheManager.dll | 8 | File Package Services Job Manager | FpsCacheManager File System Access |
d5a9622a-d3db-414b-9c4f-edb1e90aef04 | csmanext.dll | 4 | DSM CsMan extension | enteo csman core extension |
1d286dbe-eced-431c-88b8-a2c9732e1524 | syncserv.dll | 10 | DSM sync extension | enteo syncserv core extension |
ffd85e70-16d8-4f9a-a7ee-7cb6e86b168a | nwcmext.dll | 3 | DSM Network Connection Manager Extension | enteo NWCM |
2c8aaf26-9c78-4938-ac19-095f755b887c | RemoteExt.dll | 1 | DSM Remote Control mgmtagnt extension | HEAT Remote Interface |
When starting mgmtagnt.exe
with the additional command line argument /run=ersupext.dll
, which is the case for the
Windows Service Ivanti DSM Runtime Service
, the following interfaces are available:
UUID | DLL | Method Count | Name | Annotation |
---|---|---|---|---|
31a56fe8-5465-41e0-81c6-65657558f080 | ersupext.dll | 2 | DSM Runtime Service | enteo named pipe installer launch |
a3ff6366-1fd8-4330-9960-d10db0958b30 | ersupext.dll | 3 | DSM Runtime Service | enteo fastinstall launch |
f2c9e37c-abf4-4f79-8845-bd227482df3c | ersupext.dll | 1 | DSM Runtime Service | enteo fastinventory launch |
a2bb67f1-bb49-4cb6-a443-f37dc6fb25ce | ersupext.dll | 3 | DSM Runtime Service | enteo MP installation launch |
36368a9b-24ac-438c-8c49-bbab6b2761fb | ersupext.dll | 5 | DSM Runtime Service | enteo task execution |
b008df9c-0f4d-44fa-9083-9480855572b8 | ersupext.dll | 4 | DSM Runtime Service | enteo Windows Update Proxy Interface |
fd71818d-c76b-458b-bcd3-40963e9db3bf | ersupext.dll | 14 | DSM Runtime Service | Interface for the DSM Installer |
In a normal setup all managed hosts should run both services simultaneously.
Some of the available interface annotations appear quite interesting from an attacker’s point of view. Annotations like
enteo task execution
or LocalJobManager Interface
are promising candidates for remote code
execution or local privilege escalation bugs.
To call a method within any of the interfaces, an endpoint must be specified to act as the connection sink. The service binary creates a number of these endpoints at startup and the names reflect a few of the past ownership changes of DSM. Most endpoints are only accessible locally (ncalrpc), but some named pipe endpoints (ncacn_np) are also available by default.
The following table contains some examples for local and remote accessible endpoints:
Endpoint Type | Endpoint Name |
---|---|
ncalrpc | ENTEO_LJM_LRPC |
ncalrpc | HEAT_REMOTE_LRPC |
ncacn_np | \pipe\8107ec4c-4c22-42e9-872e-0bd2d534a4a4 |
ncacn_np | \pipe\a84c8520-e950-415f-b847-f96666710744 |
Attack Surface of DSM RPC
Enumeration of the security mechanisms configured is required to evaluate the attack surface on the available DSM RPC interfaces. Using rpv-web, this can be achieved by looking at:
- The authentication information that is registered for the RPC server
- The security callbacks that are registered for each interface
- The RPC flags that are configured for each interface
The key takeaways from this analysis are:
- Both mgmtagnt processes register an authentication provider for the RPC server (
RPC_C_AUTHN_WINNT
(NTLM
)) - All RPC interfaces have security callbacks configured
- No ACL’s are applied on the interfaces
- Apart from
RPC_IF_ALLOW_CALLBACKS_WITH_NO_AUTH
, no other flags are in use
Registering an authentication provider enables the RPC server to identify the caller and to perform access control checks based on the caller’s identity. Such checks can be implemented by specifying a SecurityDescriptor or SecurityCallback during the interface registration.
When a SecurityDescriptor was specified, the caller’s access token gets checked against it before the RPC call gets dispatched. The SecurityCallback gets invoked after ACL checks have passed, but before calling the actual RPC method. This allows the server to abort the RPC call if certain requirements are not met.
If a security callback is registered for an interface, all clients need to be authenticated.
This can be accomplished by calling RpcBindingSetAuthInfoEx
, which creates an authenticated binding.
An exception for this requirement are RPC interfaces that are configured with the
RPC_IF_ALLOW_CALLBACKS_WITH_NO_AUTH
flag, here no authentication is needed. Nevertheless, remote named pipe
connections still require a valid local / domain account on newer Windows systems.
In case of DSM there are five different security callbacks, all being defined in mgmtagnt.exe
.
These can be summarized as follows:
- Ensure client connects via ALPC (ncalrpc) or Named Pipe (ncacn_np), impersonate the client and revert to self (3 Interfaces)
- Ensure client connects via ALPC, impersonate the client and revert to self (14 interfaces)
- Ensure client connects via Named Pipe, impersonate the client and revert to self (1 interface)
- Ensure client connects via ALPC, enforce
RPC_C_AUTHN_LEVEL_PKT
or higher (8 interfaces) - Compare chosen RPC endpoint against a hard-coded list of allowed endpoints for the called interface (2 interfaces)
Notably none of the above security callbacks takes its decision based on the method that gets invoked by the caller. This means that callback cache abuses as demonstrated by the colleagues at Akamai are not possible.
Apart from these callback functions, DSM also implements some filtering on the method level. This is mainly done to verify that calls were dispatched using the intended endpoint (e.g., ALPC instead of named pipes). Some methods also implement security checks on the callers identity (e.g., checking that the caller is member of the local administrator group).
You might wonder why there are checks for the endpoint used, aren’t interfaces not already exclusively bound to a single endpoint? Nope, due to a great architecture decision back in the day all RPC interfaces spawned by a process can be called by all available endpoints.
The Curious Case of the CsMan Extension
From an attacker’s perspective, RPC interfaces that can be accessed remotely are of course
the most interesting ones. If you paid close attention to the security callbacks discussed in
the previous chapter, you will have noticed that among the 21 exposed RPC interfaces, only four
pass the security callback when the connection is established through a named pipe.
One of them is the DSM CsMan extension
interface that exposes the following methods:
[
uuid(d5a9622a-d3db-414b-9c4f-edb1e90aef04),
version(1.0)
]
interface d5a9622ad3db414b9c4fedb1e90aef04
{
error_status_t Proc0(
/* binding */
handle_t arg0,
[in] long arg1,
[in][string] wchar_t* arg2,
[out] long* arg3,
[in] long arg4,
[in] long arg5,
[in][string] wchar_t* arg6,
[in][string] wchar_t* arg7
);
error_status_t Proc1(
/* binding */
handle_t arg0,
[out] long* arg1,
[out] long* arg2
);
error_status_t Proc2(
/* binding */
handle_t arg0,
[in] long arg1
);
error_status_t Proc3(
/* binding */
handle_t arg0,
[in][string] wchar_t* arg1,
[in] long arg2,
[out] hyper* arg3
);
}
The interface is protected by the security callback number 1 (see above). It is thus verified that the connection was made through a named pipe or ALPC and that it is possible to impersonate the calling user on the target host. This requirement can be fulfilled by using an arbitrary domain user or a local computer account.
After creating a snapshot with rpv-web it can be imported in rpv-ghidra. This enables a quick navigation between the different RPC methods exposed by this interface.
While analyzing the available subroutines, the following log call in Proc3 quickly caught our attention:
FUN_623cf1c7(local_1c,L"csmanext",L"UpgradeMSI",1,0,(wchar_t *)0x0,0,L"ClientServices\\",0);
Based on the strings passed, it seems that the method can be used to trigger some kind of upgrade probably via an MSI file. Sounds interesting, especially if the MSI path can be provided by the caller. Digging a bit deeper confirmed that this is exactly the case. It also showed that the method was meant to upgrade the DSM agent itself.
So you say we can just pass the location of a malicious MSI package and the service happily installs it for us? Not quite. With good reason, Ivanti has put some validation in place and does not allow the installation of arbitrary packages.
The following listing outlines the function logic as pseudocode (before the fix):
function UpgradeMSI(msi_path)
{
msiversion = getVersion(msi_path)
if (msiversion <= getCurrentDSMClientVersion())
{
fail
}
signaturevalid = checkSignatureIsValidAndFromIvanti(msi_path)
if (signaturevalid == FALSE)
{
fail
}
CreateProcess(msiexec, $msi_path)
}
As visible above, all checks are performed against the provided MSI path, which is fully under the attackers control. Looking at the logic, an attacker might be able to selectively bypass both checks, if he could return different files for each file read operation (classical time-of-check (TOC) vs. time-of-use (TOU)). As the MSI path is fully controllable, there are at least two ways (SMB and OPLocks) to achieve this.
Opportunistic Locks combined with Junctions provides a relibale way of exploiting TOCTOU vulnerabilities locally, as both can be created with normal user permissions. For more information, we recommend reading the blog posts Exploiting TOCTOU vulnerability using OpLock and Junctions by Luca Barile and A Link to the Past by James Forshaw.
Specifying a remote server through a UNC path causes the DSM agent to
retrieve the MSI via the SMB protocol. An attacker could thus set up a custom
SMB server that exchanges the target file between the different file reads.
This way, the vulnerabilitiy can even be used to achieve remote code execution
on all systems running the Ivanti DSM Core Services
.
This includes at least all managed hosts and the central software depot.
Proof of Concept
The following demo demonstrates abusing the vulnerability for remote code execution.
The exploit was executed from the workstation ws131812
, where the LANMANSERVICE
was disabled to allow starting a custom service on port 445. The target worksation
ws291812
is just a managed DSM client with version 7.4.4.5362 (2020.2) installed:
You can find the source code for the exploit within the DSM MSI project of our dsm Visual Studio Solution. The source code for the required TOCTOU SMB server can be found within the TOCTOU SMB Server project. However, the actual hard work on this one was done by James Forshaw, who already implemented a TOCTOU SMB for one of his research projects and Tal Aloni, the maintainer of SMBLibrary.
Note that the exploit was tested for DSM client version 7.4.4.5362 (2020.2) and might
not work directly for other versions if the number of file reads differ. An MSI package
stating that it contains a newer DSM client version can be easily crafted using
a one-liner like: cat dsm.msi | sed 's/7.4.5362/8.0.0000/g' | sponge dsm8.msi
.
The Fix
This issue and another LPE reported by us in March 2022 was silently fixed by the vendor in June 2022 with version 7.5.0.5559 2022.1.1 Service Update 1. The MSI package is now copied to a protected directory on the local machine and subsequent checks are performed on the immutable copy.
That being said, you may still find this RPC method useful e.g. for coercing authentication or
copying files to the target host. The curious among you might even want to check the usage of the
registry key HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\NetSupport\NetInstall\trust
for lateral
movement :-).
Who Could Have Guessed It?
The second vulnerability we want to look at is a local privilege escalation bug.
It may come as a surprise to you, but this vulnerability was contained within
the RPC interface labeled enteo task execution
. The following listing
shows the first method from this interface:
[
uuid(36368a9b-24ac-438c-8c49-bbab6b2761fb),
version(1.0)
]
interface 36368a9b24ac438c8c49bbab6b2761fb
{
error_status_t Proc0(
/* binding */
handle_t arg0,
[out][context_handle] void** arg1,
[in] long arg2,
[in][string] wchar_t* arg3,
[in][string] wchar_t* arg4,
[in][string] wchar_t* arg5,
[in] long arg6,
[in] long arg7,
[out] long* arg8
);
...
}
Combining the interface description enteo task execution
with an
argument structure that contains three strings might make you think:
arg3
:lpApplicationName
arg4
:lpCommandLine
arg5
:lpCurrentDirectory
But it cannot be that easy, right? Well, the following screenshot with the functions’ call tree may answer that question:
The RPC method is indeed just a pass through to CreateProcessW
. Fortunately,
the security callback of this RPC method verifies that the call was made
through an ncalrpc endpoint, which makes this vulnerabilitiy only exploitable
locally.
Proof of Concept
The following demo shows how the vulnerability could be abused for local privilege escalation:
You can find the source code for the exploit within the DSM Exec project of our dsm Visual Studio Solution.
The Fix
After initial discovery, the vulnerability was immediately reported to Ivanti.
CVE-2023-28129
was assigned, and the issue was fixed in DSM 2022.2 SU3.
To fix the issue, Ivanti added another check within the RPC method. It now checks
if the caller is a member of the local administrators group. If this is not the
case, the call is rejected. Apart from that, nothing has changed, meaning that local
administrators can still start arbitrary processes as child of the mgmtagnt.exe
process.
Decrypting Credentials
Finally, we will take a look on how to decrypt credentials via RPC. If you were able to follow up to this point, decrypting credentials will be an easy task, as you only have to find the correct RPC method.
Since credentials are stored within the ICDB it is likely
that the desired RPC interface is defined in icdbext.dll
,
which exposes three RPC interfaces with a total of 31 methods.
The decryption method is Proc19
from the following interface:
[
uuid(5a622071-c10c-4160-8327-20e943f41644),
version(1.0)
]
interface 5a622071c10c4160832720e943f41644
{
...
error_status_t Proc19(
/* binding */
handle_t arg0,
[in][string] char* arg1,
[in][string] char* arg2,
[out][ref][string] char** arg3,
[in] long arg4
);
}
In case it was unclear as to how we identified this:
The DSM Decrypt project of our dsm Visual Studio solution contains a proof of concept for decrypting credentials via RPC. Suffice to say that this tool is only for educational purposes and must not be used for any malicious activities.
RPC Related Recommendations
In this section we demonstrated attacks on the three DSM RPC interfaces:
d5a9622a-d3db-414b-9c4f-edb1e90aef04
36368a9b-24ac-438c-8c49-bbab6b2761fb
5a622071-c10c-4160-8327-20e943f41644
Since these interfaces are used internally by DSM, it is very unlikely that other applications interact with them. Any invocations from executables not part of the DSM ecosystem, might thus be seen as IOC.
Additionally, it might make sense to monitor mgmtagnt.exe
for unusual activity,
as this is the executable registering all the RPC interfaces. Due to the presence
of RPC interfaces like enteo task execution
, it is expected that this process
may create child processes from time to time. However, the amount of legitimately
started child processes is probably limited and can be excluded after a short
learning period.
From what we have seen so far, we believe that there are more attack opportunities within the huge RPC interface pool of DSM. Adding a SIEM rule to capture connections to the RPC endpoints from unknown processes / atypical hosts would therefore probably be a good idea.
Using DSM Software Distribution for Lateral Movement
When talking about cool hacks, people usually associate them with a shady hacker sitting in a dark room and performing seemingly magic attacks like decrypting protected credentials or getting remote code execution via RPC. During red team assessments however, it turns out that often times, the most scary attacker is the one that is just capable of using enterprise software legitimately.
Using software distribution solutions for lateral movement is of course not a novel idea. First resources on this topic go back to 2012, where the folks at TrustedSec demonstrated malicious software deployment using SCCM during their Owning One to Rule Them All Defcon talk. In the last couple of years, software distribution abuse is seemingly getting more and more popular, which can probably be explained by the increase in security research and tool development in that area. The expanding usage of cloud based software deployment solutions like Microsoft Intune also make this vector interesting for entry point exploitation.
Software deployment via DSM is a little bit different. There is no publicly documented API, no PowerShell modules shipped by the vendor and, to the best of our knowledge, no previous research discussing this topic. The technique we demonstrate in this blog is probably not the best and has some obvious opsec weaknesses. That being said, we compromised a double digit number of environments using this technique so far while staying undetected :).
Using the DSM Console
When connecting to a DSM depot server via SMB, you will find an application called
dsmc.exe
. This is the DSM Console, that can be used to configure and manage
the DSM environment. The application can be started by any user account, but
the number of menus and configuration options you see will vary depending on
your permissions. When starting the application, make sure you execute it directly
from the network drive (even with local access to the DSM depot, it will not start
unless you execute it from a network drive).
In the following we assume that we have a user with Supervisor
privileges.
This is the most powerfull role you can have in DSM and it is not uncommon to find
such a user with decryptable credentials within the ICDB. If you have access to
the DSM database, you can enumerate the users with supervisor privileges by investigting
the CMPG_USER
, CMPG_EXTERNALGROUP
and CMAS_ROLEEXTERNALGROUPS
tables.
If your user has sufficient privileges, you may even be able to access the
Manage Roles
menu in the DSM config:
Creating an eScript Package
In the following we describe how to create a new software package within the
DSM console. Creating such a package and deploying it to selected managed endpoints
has been proven a valuable lateral movement technique in DSM managed environments.
DSM supports an number of package types that might be useful depending on the situation.
We will stick to the eScript Package
type, which is basically a simple file deployment
combined with an installation script.
To create such a package, right click the Application Library
item within the
directory tree view in the button left of the DSM console. Within the context menu, click Automate -> Distribution of Files
:
This opens the Content Package Wizard, where you first have to select a name
for your newly created package. In this example, we choose the generic name
CWPatchHelper
and continue to the file selection dialog. This dialog requests
you to upload all the files that should be distributed with your package. We
go ahead and upload an innocent executable together with a nice logo.
After confirming the upload you will be taken to the Packaging Workbench. Within the Specify Platforms menu, one may configure on which platforms your package can be installed. The Define Execution Settings menu can be used to configure in which context (user or system) your package needs to be installed and whether a reboot after installation is required.
Now it is time to write the eScript
. Clicking Edit Script takes you
to the script editor where you can provide an installation script for
your package. To the best of our knowledge eScript
is a proprietary
scripting language used by DSM. Using the displayed command on the left
hand side is a great way to get started. Since we created the
package using the Distribution of Files
package type, the eScript
already contains a line that deploys the uploaded files to the same
location on each node the package gets deployed. We supplement this
initial instruction by a line that runs our executable during
installation. Additionally, we provide some uninstall commands that
remove the uploaded files from the endpoint:
As a last step it is recommended to specify some compliance checks. These are used by DSM to determine whether your package was installed successfully. For our current case, we can simply check whether the desired files were deployed successfully:
After saving the package, it should appear within the Objects view of the Application Library. The next steps in a typical DSM package lifecycle would be:
- Prepare the package for distribution
- Distribute the package to the depot servers
- Release the package
- Assign the package to targeted endpoints
However, for red team assessments, it is usually sufficient to create a Pilot Installation.
Creating a Pilot Installation
DSM’s pilot installation feature is meant to test packages before an actual deployment. It still requires you to distribute the package but creating a release is not required. Within the pilot installation you can still assign the package to target users or computers which makes it the preferred option for lateral movement.
After selecting the package, hitting Prepare Distribution on the right hand side will prepare the distribution of the package. After the button changes color to gray, the Distribution Setup button can be clicked. This deploys the package to the depot servers. Depending on the number of available depots, distribution can take a while. You can always check the status clicking the View Distribution Status button:
After the package has been distributed we are ready to hit the Execute pilot installation button. An assignment menu pops up where one can select managed users or computers the pilot installation should be applied to:
After selecting the target, one can continue with basic or advanced configurations. The advanced configurations enable you to control whether an installation can be posponed or interrupted by the user. It also allows control of whether maintainance intervals are respected.
After the pilot installation dialog has finished, DSM creates a new policy within the application Software Policies view. Due to the defined compliance check, it is now possible to monitor whether the package was installed successfully:
Software distributed by DSM is normally installed silently in the background.
However, by starting the Software Shop application, it is possible to view
the installation process. After a package was installed, it gets listed
within the My Software section of the Software Shop application. The
following screenshot shows the view of the client after the CWPatchHelper
application was installed. As a result of this installation, the desktop
wallpaper was changed (which is of course one of the more harmless things
you can do ;D).
Accessing Packages via SMB
It is worth noting that, as in the case of SCCM, available packages can be
enumerated and inspected via SMB from a DSM depot server. During creation
packages are assigned a unique ID and are stored on the DSM depots under
\\<DEPOT>\Work\Master\Projects\ID
. The example package from above received
ID 853
and the its contents can be found within the respective
folder:
user@host:~/$ smbclient.py cw:redacted@depot.redacted.local
Type help for list of commands
# use DSM$
# cd Work\Master\Projects\853
# ls
drw-rw-rw- 0 Fri Apr 4 21:53:29 2025 .
drw-rw-rw- 0 Fri Apr 4 21:53:29 2025 ..
drw-rw-rw- 0 Fri Apr 4 16:29:44 2025 ALLUSERSPROFILE
-rw-rw-rw- 324 Fri Apr 4 22:03:53 2025 CompProject.status
-rw-rw-rw- 50 Fri Apr 4 21:53:29 2025 Project.jdf
-rw-rw-rw- 324 Fri Apr 4 22:03:53 2025 Project.status
drw-rw-rw- 0 Fri Apr 4 17:09:47 2025 rev
-rw-rw-rw- 453 Fri Apr 4 16:29:45 2025 Script.BA1
-rw-rw-rw- 341 Fri Apr 4 16:43:44 2025 Script.inc
# cd ALLUSERSPROFILE
# ls
drw-rw-rw- 0 Fri Apr 4 16:29:44 2025 .
drw-rw-rw- 0 Fri Apr 4 16:29:44 2025 ..
-rw-rw-rw- 64000 Fri Apr 4 16:26:49 2025 CWPatchHelper.exe
-rw-rw-rw- 28699 Fri Apr 4 16:26:52 2025 logo.png
# Bye!
There is no access control on such packages, meaning that if you can access a DSM depot, you can inspect all software available on it. Finding credentials or other sensitive information this way is not rare and inspecting the available DSM packages can be worth the effort.
Recommendations for DSM Software Deployments
The most effective step to prevent malicious software deployments is of course to strictly control which accounts are able to deploy software and to protect them sufficiently. To supplement such protections, it is recommended to implement monitoring rules that create notifications when new DSM packages get created.
DSM packages are created by sending SMB and SOAP messages to the depot server.
In theory, attackers could perform these steps manually, but it is likely that
dsmc.exe
is used to simplify the process. Monitoring invocations of dsmc.exe
from uncommon sources could also be an effective rule to detect malicious
activity.