Implement dynamic loading of Tor Hidden Service Keys

[fpietrosanti]

Following the implementation of Tor's Ticket https://trac.torproject.org/projects/tor/ticket/5976 this ticket is to implement proper txtorconn functionalities for:

• Loading a Tor Hidden Service Key via Tor CP
• Querying the Tor Hidden Service Hostname via Tor CP

This ticket is also referenced by globaleaks/APAF#27

[meejah]

You've filed this issue for whenever Tor implements a control-protocol method of setting private keys for hidden services? Or does such an option already exist? Presumably this could be done with a new option like HiddenServicePrivateKey or something that contained the key data.

Would a workaround not be to put the tor config -- or at least the hidden service dir -- in a ramdisk (encrypted, even)? Or in a file wrapped by a FUSE encrypted filesystem?

[fpietrosanti]

Hi Meejah,

yes, this ticket is for future reminder about all the projects that will require actions when the Tor ticket will get implemented.

The workaround you refer can work only with:
a) administrative privilege access (require kernel loading privileges)
b) touching kernel (require FUSE kernel module)
c) with high implementation complexity for cross-platform uses (a lot of different code from different platform)

The concept was to implement a generic method to load keys via Tor CP into Tor.

The pain is that it's required to make a TorCP change proposal, submit it, have review, then implement it., then have it reviewed.

[fpietrosanti]

Wait, i got an idea.

What if we make Txtorconn:

• generate TorHS key
• save them into an encrypted db (sqlite/sqlcipher)
• create a FIFO (or named pipe on windows) where TorHS key and "hostname" files should be
• when tor goes for reading it, it return the content extract from the db (sqlite/sqlcipher)

That way txtorconn would be able to provide an abstraction layer over the weak handling of TorHS keys, basically using mkfifo (fifo / named pipes) as a filesystem I/O bridge between txtorconn application logic (providing the TorHS key) and Tor binary.

So the TorHS private key will never be written to the filesystem in unencrypted format, like tor currently does.

What do you think, it's worth experimenting it?

[fpietrosanti]

On unix the standard os module os.mkfifo already does provide the facility to create FIFO:
http://docs.python.org/library/os.html

On windows is it possible to use python to create named pipes (windows equivalent of unix fifo):
http://stackoverflow.com/questions/2922185/python-os-mkfifo-for-windows/2922849#2922849

Additionally on Windows named pipes can be created with gnuwin32 utils mkfifo create named pipes: http://gnuwin32.sourceforge.net/packages/coreutils.htm

[fpietrosanti]

Example in using Named Pipes on Windows with Python for Wireshark captures http://wiki.wireshark.org/CaptureSetup/Pipes

[meejah]

Named pipes using Twisted http://twistedmatrix.com/pipermail/twisted-python/2003-May/004111.html

There's also txnamedpipes (for windows only, it looks like).

[fpietrosanti]

That ticket is blocked by defect https://trac.torproject.org/projects/tor/ticket/6044 as reported by meejah

[meejah]

what's going on is that file_status() returns FN_ERROR on FIFOs, and (even if it didn't) read_file_to_str() uses fstat() to figure out the size beforehand, which for FIFOs is 0. I am working on a patch for this, although I don't have a windows machine so don't know how to make it work there (or maybe it already will!).

The "no such file or directory" stuff from the pad is a red herring as that's leftover from other stat or open calls as tor starts up.

[fpietrosanti]

Will provide a windows machine to test integration with Windows Named Pipes both of Tor and Txtorcon patchset

[fpietrosanti]

On Tor there's someone that recently pushed a patch to be reviewed doing that feature for handling TorHs via TorCP: https://trac.torproject.org/projects/tor/ticket/6411#comment:6

[david415]

I think this Tor Project ticket is related to this github issue:
https://trac.torproject.org/projects/tor/ticket/1949

Maybe we can together work on the little-t tor code changes... like we did last time!

[meejah]

Actually, see https://trac.torproject.org/projects/tor/ticket/6411 it seems there is other interest in this, and a patch in-progress to add controller commands to allow applications to directly add hidden-service keys they manage themselves.

There would be txtorcon work when 6411 is done: augmenting TorConfig to generate keys and use the new commands to add/remove these hidden services.

One open question would be whether it makes sense to have TorConfig.hiddenservices contain both "permanent" and "ephemeral" hidden services, or to have two lists...

[fpietrosanti]

@meejah I think that once the Tor-code to load with Tor CP is Ephemeral itself, TxTorCon shall provide ephemeral support for key too, leaving to the application logic the handling of the key material.
What would be useful, for sure, is the support for the key generation in a "safe enough" way, so the application can just ask TxTorCon to generate a new key or to load an existing one previously generated.

[Yawning]

@fpietrosanti Hopefully generating RSA keys is well understood by this point. The format the WIP code expects is just a Base64 string containing the PKCS#1 style DER encoded private key, which is fairly ubiquitous (It's the text that appears between "-----BEGIN RSA PRIVATE KEY-----" and "-----END RSA PRIVATE KEY-----" with the newlines stripped). As I noted in the bug, there's sample code at https://github.com/yawning/orkey that does this correctly (though Go provides both RSA key generation and an ASN.1 DER encoder as part of the runtime).

Open design questions (some mirrored from the tor side bug, though if people are talking here, I might as well ask again):

• "250 OK" is a bit terse as far as responses go. Should the onion service address be provided as well? (I'm somewhat inclined to think that this is an application problem, since the onion address is trivially derived from the private key).
• Should the ephemeral services disapear on config reload (Eg: SIGHUP?). The code in my branch will tear them all down, because that was the easiest thing to implement, but persisting them isn't much harder.
• (Long term) Authenticated HSes should be supported, but won't be in the initial revision. Someone should come up with a sane design for this (Tor bug 1949 indicates that people thought about this more than I have).

There's no particular rush from my side, since this missed the 0.2.6.x feature freeze date, and it will be a while before 0.2.7.x.

[meejah]

See branch https://github.com/meejah/txtorcon/tree/yawning-feature-6411 (yawning-feature-6411) which can be used along with @Yawning's branch for Tor's #6411 (https://trac.torproject.org/projects/tor/ticket/6411) to try all this out.

[fpietrosanti]

I am so happy! :-) we started looking at this issue in august 2012, with this branch released under Tor 0.2.7 it will finally become a reality!
Tnx @meejah and @Yawning !

[fpietrosanti]

@meejah has this feature been merged in txtorcon? As we're now finally considering using it in GlobaLeaks /cc @evilaliv3

[meejah]

Yes, you can pass keys directly to a new enough Tor with the ADD_ONION command via EphemeralHiddenService https://github.com/meejah/txtorcon/blob/master/txtorcon/torconfig.py#L906

The API will likely change slightly in 1.0 as I've unified all the various hidden-service options in that branch. But, nothing drastic.