TLSleuth already supports several transport modes for retrieving TLS certificate and handshake details:

Those transports gave the module a useful foundation, but they are not all the same kind of protocol flow. SMTP, IMAP, and POP3 use line-oriented plaintext negotiation before TLS starts. RDP is different: it needs binary negotiation before the connection can be upgraded into the TLS path.

Before adding RDP certificate retrieval, the transport layer needed a small internal cleanup. The goal was not to expose new public parameters yet. The goal was to make the existing transport pipeline ready for a binary protocol without mixing RDP packet handling into the current text protocol helpers.

What Changed

The recent refactor focused on three areas:

  1. structured transport negotiation metadata
  2. clearer transport option modeling
  3. a separate binary protocol helper boundary

These changes are internal. The public output contracts for Get-TLSleuthCertificate and Test-TLSleuthProtocol remain unchanged.

Transport Negotiation Now Has An Internal Result

Invoke-TlsTransportNegotiation previously performed the transport-specific work and then returned nothing meaningful to its caller. That was acceptable while the only job was “do any plaintext negotiation and continue.”

For RDP, that will not be enough. A future RDP transport may need to record negotiation details such as the selected security mode or a protocol-specific failure reason.

TLSleuth now has a dedicated helper for that internal metadata:

New-TlsTransportNegotiationResult

It returns a TLSleuth.TransportNegotiationResult object with these fields:

Invoke-TlsTransportNegotiation now returns that object for current transports. For example:

The public commands intentionally suppress this object:

$null = Invoke-TlsTransportNegotiation `
    -Transport $context.Transport `
    -Connection $connection `
    -Options $context.TransportOptions

That keeps existing user-facing output stable while making the transport layer more useful internally.

Transport Options Are Now Structured

The next problem was option shape. SMTP already had a protocol-specific setting: SmtpEhloName. RDP is expected to need its own protocol-specific options later, and adding those directly beside shared options would make TransportOptions harder to reason about.

TLSleuth now uses:

New-TlsTransportOptionSet

The option set separates shared options from protocol-specific options:

TransportOptionSet
  Common
    TimeoutMs
  SmtpStartTls
    EhloName
  Rdp
    RequestedSecurityProtocol

New-TlsOperationContext builds this option set as part of the shared command context. Public commands still pass one TransportOptions object into the dispatcher, but the internals now have a clearer place for future RDP-specific configuration.

SMTP EHLO behavior is unchanged. If no EHLO name is supplied, the SMTP negotiation path still falls back through the existing hostname behavior.

Binary Helpers Stay Separate From Text Helpers

The existing explicit TLS transports share text protocol helpers:

Those helpers are the right fit for SMTP, IMAP, and POP3. They are not the right fit for RDP.

The RDP path will need byte-oriented packet work, so TLSleuth now has separate binary helpers:

Read-BinaryProtocolData
Send-BinaryProtocolData

Read-BinaryProtocolData reads an exact byte count from a stream. If the stream ends early, it throws. Send-BinaryProtocolData writes raw bytes and flushes the stream without text encoding.

The unit tests use representative binary data, including bytes shaped like an RDP-style packet header, to prove that binary data is not routed through ASCII line handling.

Updated Flow

At a high level, the transport pipeline now looks like this:

flowchart TD
    PublicCommand["Get-TLSleuthCertificate / Test-TLSleuthProtocol"]
    OperationContext["New-TlsOperationContext"]
    OptionSet["New-TlsTransportOptionSet"]
    Connect["New-TlsConnectionContext"]
    Transport["Invoke-TlsTransportNegotiation"]
    Result["New-TlsTransportNegotiationResult"]
    TextHelpers["Text helpers: SMTP / IMAP / POP3"]
    BinaryHelpers["Binary helpers: future RDP path"]
    Handshake["Start-TlsHandshake"]
    Output["Public TLSleuth result object"]

    PublicCommand --> OperationContext
    OperationContext --> OptionSet
    PublicCommand --> Connect
    PublicCommand --> Transport
    Transport --> TextHelpers
    Transport -. future RDP transport .-> BinaryHelpers
    Transport --> Result
    PublicCommand --> Handshake
    PublicCommand --> Output

The important part is the separation. Text transports remain text transports. Binary transports get their own byte-oriented helper path.

What This Does Not Do Yet

This work does not add a public RDP transport option yet.

There is no supported command like this yet:

Get-TLSleuthCertificate -Hostname server.example.com -Port 3389 -Transport Rdp

That is still future work. The public -Transport parameter still supports the existing transport values.

This refactor also does not implement RDP packet construction, RDP negotiation response parsing, or RDP-specific failure categorization. It only creates the internal shape those pieces can plug into.

Why Do This Before RDP?

RDP support will be easier to review if it only adds RDP behavior. Without this prep work, the RDP feature would have mixed several unrelated changes into one larger patch:

Splitting the prep work out keeps the future RDP implementation smaller and makes regressions easier to spot.

Validation

The refactor is covered by focused unit tests:

The validation commands used for this work were:

Invoke-Build UnitTest -File .\TLSleuth.build.ps1
Invoke-Build Analyze -File .\TLSleuth.build.ps1
Invoke-Build Build -File .\TLSleuth.build.ps1
Invoke-Build ValidateReleaseMetadata -File .\TLSleuth.build.ps1

Summary

This change prepares TLSleuth for RDP by making the transport layer more explicit:

The next step is to add the actual RDP-aware transport path: build the RDP negotiation packet, parse the response, select the correct security path, and then hand the stream into the existing TLS handshake and certificate inspection flow.