Osmocom Planet Osmocom
Open Source Mobile Communications

January 31, 2016

Harald "LaForge" Welte: On the OpenAirInterface re-licensing

In the recent FOSDEM 2016 SDR Devroom, the Q&A session following a presentation on OpenAirInterface touched the topic of its controversial licensing. As I happen to be involved deeply with Free Software licensing and Free Software telecom topics, I thought I might have some things to say about this topic. Unfortunately the Q&A session was short, hence this blog post.

As a side note, the presentation was actually certainly the least technical presentation in all of the FOSDEM SDR track, and that with a deeply technical audience. And probably the only presentation at all at FOSDEM talking a lot about "Strategic Industry Partners".

Let me also state that I actually have respect for what OAI/OSA has been and still is doing. I just don't think it is attractive to the Free Software community - and it might actually not be Free Software at all.

OpenAirInterface / History

Within EURECOM, a group around Prof. Raymond Knopp has been working on a Free Software implementation of all layers of the LTE (4G) system known as OpenAirInterface. It includes the physical layer and goes through to the core network.

The OpenAirInterface code was for many years under GPL license (GPLv2, other parts GPLv3). Initially the SVN repositories were not public (despite the license), but after some friendly mails one (at least I) could get access.

I've read through the code at several points in the past, it often seemed much more like a (quick and dirty?) proof of concept implementation to me, than anything more general-purpose. But then, that might have been a wrong impression on my behalf, or it might be that this was simply sufficient for the kind of research they wanted to do. After all, scientific research and FOSS often have a complicated relationship. Researchers naturally have their papers as primary output of their work, and software implementations often are more like a necessary evil than the actual goal. But then, I digress.

Now at some point in 2014, a new organization the OpenAirInterface Software Association (OSA) was established. The idea apparently was to get involved with the tier-1 telecom suppliers (like Alcatel, Huawei, Ericsson, ...) and work together on an implementation of Free Software for future mobile data, so-called 5G technologies.

Telecom Industry and Patents

In case you don't know, the classic telecom industry loves patents. Pretty much anything and everything is patented, and the patents are heavily enforced. And not just between Samsung and Apple, or more recently also Nokia and Samsung - but basically all the time.

One of the big reasons why even the most simple UMTS/3G capable phones are so much more expensive than GSM/2G is the extensive (and expensive) list of patents Qualcomm requires every device maker to license. In the past, this was not even a fixed per-unit royalty, but the license depended on the actual overall price of the phone itself.

So wanting to work on a Free Software implementation of future telecom standards with active support and involvement of the telecom industry obviously means contention in terms of patents.

Re-Licensing

The existing GPLv2/GPLv3 license of the OpenAirInterface code of course would have meant that contributions from the patent-holding telecom industry would have to come with appropriate royalty-free patent licenses. After all, of what use is it if the software is free in terms of copyright licensing, but then you still have the patents that make it non-free.

Now the big industry of course wouldn't want to do that, so the OSA decided to re-license the code-base under a new license.

As we apparently don't yet have sufficient existing Free Software licenses, they decided to create a new license. That new license (the OSA Public License V1.0 not only does away with copyleft, but also does away with a normal patent grant.

This is very sad in several ways:

  • license proliferation is always bad. Major experts and basically all major entities in the Free Software world (FSF, FSFE, OSI, ...) are opposed to it and see it as a problem. Even companies like Intel and Google have publicly raised concern about license Proliferation.
  • abandoning copyleft. Many people particularly from a GNU/Linux background would agree that copyleft is a fair deal. It ensures that everyone modifying the software will have to share such modifications with other users in a fair way. Nobody can create proprietary derivatives.
  • taking away the patent grant. Even the non-copyleft Apache 2.0 License the OSA used as template has a broad patent grant, even for commercial applications. The OSA Public License has only a patent grant for use in research context

In addition to this license change, the OSA also requires a copyright assignment from all contributors.

Consequences

What kind of effect does this have in case I want to contribute?

  • I have to sign away my copyright. The OSA can at any given point in time grant anyone whatever license they want to this code.
  • I have to agree to a permissive license without copyleft, i.e. everyone else can create proprietary derivatives of my work
  • I do not even get a patent grant from the other contributors (like the large Telecom companies).

So basically, I have to sign away my copyright, and I get nothing in return. No copyleft that ensures other people's modifications will be available under the same license, no patent grant, and I don't even keep my own copyright to be able to veto any future license changes.

My personal opinion (and apparently those of other FOSDEM attendees) is thus that the OAI / OSA invitation to contributions from the community is not a very attractive one. It might all be well and fine for large industry and research institutes. But I don't think the Free Software community has much to gain in all of this.

Now OSA will claim that the above is not true, and that all contributors (including the Telecom vendors) have agreed to license their patents under FRAND conditions to all other contributors. It even seemed to me that the speaker at FOSDEM believed this was something positive in any way. I can only laugh at that ;)

FRAND

FRAND (Fair, Reasonable and Non-Discriminatory) is a frequently invoked buzzword for patent licensing schemes. It isn't actually defined anywhere, and is most likely just meant to sound nice to people who don't understand what it really means. Like, let's say, political decision makers.

In practise, it is a disaster for individuals and small/medium sized companies. I can tell you first hand from having tried to obtain patent licenses from FRAND schemes before. While they might have reasonable per-unit royalties and they might offer those royalties to everyone, they typically come with ridiculous minimum annual fees.

For example let's say they state in their FRAND license conditions you have to pay 1 USD per device, but a minimum of USD 100,000 per year. Or a similarly large one-time fee at the time of signing the contract.

That's of course very fair to the large corporations, but it makes it impossible for a small company who sells maybe 10 to 100 devices per year, as the 100,000 / 10 then equals to USD 10k per device in terms of royalties. Does that sound fair and Non-Discriminatory to you?

Summary

OAI/OSA are trying to get a non-commercial / research-oriented foot into the design and specification process of future mobile telecom network standardization. That's a big and difficult challenge.

However, the decisions they have taken in terms of licensing show that they are primarily interested in aligning with the large corporate telecom industry, and have thus created something that isn't really Free Software (missing non-research patent grant) and might in the end only help the large telecom vendors to uni-directionally consume contributions from academic research, small/medium sized companies and individual hackers.

January 27, 2016

OpenBSC news feed: Status Report on Osmocom's 3G Support

3G is dead, you may think. From the perspective of large scale operators, that may well be the case, but this is precisely the reason why Open Source support for 3G is becoming increasingly interesting: when the focus for earning money shifts towards LTE infrastructure, the threshold for setting up 3G networks is becoming easier to surpass for everyone else.

We are implementing Iuh support in the Osmocom stack, mostly carried out by employees of sysmocom GmbH (1), with highly appreciated (yet undisclosed) external backing.

Iu support in Osmocom will allow using a femto-cell aka hNodeB as BTS, thus enabling UMTS voice (IuCS) and data (IuPS) connectivity using FOSS software from the core network right up to the femto-cell's ethernet jack.

Here is an ASCII art overview of our current aim:

        +------------+           +--------+          +----------+
 UE <-->| hNodeB     |<--Iuh---->| HNB-GW |<--IuCS-->| OsmoCSCN |
 UE <-->| femto cell |     ...-->|        |    ...-->|          |
        |            |           |        |          +----------+
        +------------+<--GTP-U   |        |
                              \  |        |          +------+           +------+
                              |  |        |<--IuPS-->| SGSN |<--GTP-C-->| GGSN |
                              |  +--------+    ...-->|      |   GTP-U-->|      |
                              |                      +------+  /        +------+
                              \_______________________________/
                      Iuh                         IuCS/IuPS
NAS                   +----+----+                 +----+----+
Non-Access Stratum    | CC | MM |                 | CC | MM |
- - - - - - - - - - - +----+----+-------+         +----+----+
                      | RANAP   |       |    H    | RANAP   |
Access Stratum        +---------+ HNBAP |    N    +---------+ - - SCCP USER SAP
                      | RUA     |       |    B    | SUA     |  \
                      +---------+-------+    -    +---------+  |
                      |        SCTP     |    G    | SCTP    |  } SIGTRAN
                      +-----------------+    W    +---------+  |
                      |        IP       |         | IP      |  /
                      +-----------------+         +---------+
UE (User Endpoint) == MS (Mobile Subscriber) == mobile device
CSCN (Circuit Switched Core Network) == OsmoNITB without BSC

(Source: http://git.osmocom.org/osmo-iuh/tree/doc/protocols_around_hnbgw.txt )

3G Data Status

The best news first: in our lab, Daniel has successfully established the first functional UMTS data link!

In other words, the two GTP paths

  • hNodeB --Iuh--> HNB-GW --IuPS--> SGSN --GTP-C--> GGSN
  • hNodeB --GTP-U--> GGSN

are already functional in a basic fashion. To test it, you would need to checkout various branches in various git repositories. These shall soon be merged to the respective master branches, but currently, that would be:

  • libasn1c: master
  • asn1c: aper-prefix
  • libosmocore: master
  • libosmo-netif: sysmocom/sctp
  • libosmo-sccp: laforge/wip
  • osmo-iuh: master
  • openbsc: daniel/gprs-iu

3G Voice Status

The voice link (IuCS) still needs some work before even the attempt of a location update will make sense. The point here is that previously, OsmoNITB would combine the roles of BSC with the MSC and further core network components. For Iuh, though, the BSC role is actually embedded in the hNodeB, and thus we are aiming to split the BSC part off of OsmoNITB.

The result will be called OsmoCSCN, CSCN meaning Circuit-Switched-Core-Network, which lives in openbsc.git/openbsc/src/osmo-cscn/ (branch sysmocom/cscn).

OsmoCSCN comprises of "only" the MSC and further CN components. It will feature an IuCS interface, as well as, eventually, a proper A-interface towards 2G BSCs, thus obsoleting the OsmoNITB at some point.

Implementing OsmoCSCN is mostly my task, and after sinking some time into training my eye on the fine line between BSC and MSC, I've finally started actual development with the dawn of 2016.

3G in a Nutshell

Let me illustrate some details of the Iu interfaces. The following is basically Harald's 3G talk at the 32c3 (2), but with the sheer abundance of complexity it can't hurt to read it in prose.

HNB-GW

The HNB-GW, i.e. the HomeNodeB Gateway, merely reads the CN-DomainIndicator from the RUA layer, which says whether the frame is for voice or data comms (IuCS or IuPS). It then sends the actual RANAP payload either to the OsmoCSCN or the OsmoSGSN. The HNB-GW is implemented in osmo-iuh/src/hnbgw.c, compiling as osmo-hnbgw, and is (mostly?) complete.

An interesting factoid is that for an hNodeB, the GTP-Control handshaking goes via the HNB-GW, while the packet user data actually goes directly to/from the hNodeB and the GGSN.

HNBAP

HNBAP is merely the protocol employed to register an hNodeB with the HomeNodeB Gateway. After HNBAP is done, the hNodeB sends RANAP-over-RUA, which the HNB-GW happily passes on to the proper consumers.

SIGTRAN

Typically, the IuCS and IuPS interfaces would talk this layering of protocols:

  CC/MM
  RANAP
  SCCP  <--- note
  M3UA  <--- note
  SCTP
  IP

We do have SCCP support in Osmocom, but so far only for "connectionless" messages (like your standard UDP datagrams). Iu now adds the need for establishing and tearing down connections (like TCP).

However, since SUA does the same as SCCP-over-M3UA and is simpler to implement, our HNB-GW talks SUA towards IuCS and IuPS:

  CC/MM
  RANAP
  SUA   <--- note
  SCTP
  IP

To support third-party MSC and SGSN components, we would either add SCCP-over-M3UA capability, or simply use an external signalling gateway that supports both M3UA and SUA (should be possible e.g. with osmo_ss7 (3)).

Various SIGTRAN implementations:

                IuCS/IuPS
                  usual
                   |     simplest
                   |       |
                   v       v
  +------+------+------+-----+
  | SCCP | SCCP |      |     |
  +------+------+ SCCP |     |
  | MTP3 | MTP3 |      |     |
  +------+------+------+ SUA |
  | MTP2 |      |      |     |
  +------+ M2UA | M3UA |     |
  | M2PA |      |      |     |
  +------+------+------+-----+
  |           SCTP           |
  +--------------------------+
  |            IP            |
  +--------------------------+

ASN1 Convolutions

RANAP, RUA and HNBAP, which make up the Iuh interface, are ASN1 encoded. Fair enough, but their ASN1 encoding uses APER, and heavily employs Information Object Classes (which basically means it wraps ASN1 encoded binary data in ASN1 IEs, with several levels of depth). In consequence, the libre asn1c compiler as-is unfortunately is not capable of generating de-/encoders for UMTS. The proprietary ffasn1c is capable of that, and we could publish the ffasn1c generated code without licensing problems, but we'd highly prefer to empower the FOSS community with the ability to modify and fix the ASN1 de-/encoders independently of proprietary software.

The great news is that Eurecom (4) has worked on supporting both APER and the nested ASN1 structures ("Information Object Classes") in asn1c, and we are able to use their solutions in a FOSS way. With some fixes added, we have both their APER support and their pythonic solution for nested ASN1 available in Osmocom's libasn1c and asn1c git repositories (5).

Another problem with the Iuh ASN1 is that various type names are identical across RANAP, RUA and HNBAP, while their encodings differ. This causes type name collisions in the code generated by asn1c, hence we have added prefixing support to our version of asn1c. This simply means that each RANAP-related type name or function begins with "ranap_", and RUA names begin with "rua_", thus avoiding any and all name collisions between those protocols. See osmo-iuh/include/osmocom/ranap/ and ../rua/.

It could be more beautiful, but the bottom line is that we now have fully free/libre support for Iuh ASN1 encodings. Cheers!

Conclusion

Osmocom is on a clear trajectory towards full 3G support, empowering remote communities and small to medium businesses worldwide. Work is ongoing, but the really hard problems have already been solved. Stay tuned!

External References

(1) http://www.sysmocom.de
(2) https://media.ccc.de/v/32c3-7412-running_your_own_3g_3_5g_network
(3) http://git.osmocom.org/erlang/osmo_ss7
(4) http://www.eurecom.fr
(5) See http://git.osmocom.org/libasn1c/ and http://git.osmocom.org/asn1c/log/?h=aper-prefix

January 18, 2016

Holger "zecke" Freyther: osmo-pcap capture on the edge and store in the center

Imagine you run a GSM network and you have multiple systems at the edge of your network that communicate with other systems. For debugging reasons you might want to collect traffic and then look at it to explore an issue or look at it systematically to improve your network, your roaming traffic, etc.

The first approach might be to run tcpdump on each of these systems, run it in a round-robin manner, compress the old traffic and then have a script that downloads/uploads it once a day to a central place. The issue is that each node needs to have enough disk space, you might not feel happy to keep old files on the edge or you just don't know when is a good time to copy it.

Another approach is to create an aggregation framework. A client will use libpcap to capture the traffic and then redirect it to a central server. The central server will then store the traffic and might rotate based on size or age of the file. Old files can then be compressed and removed.

I created the osmo-pcap tool many years ago and have recently fixed a 64bit PCAP header issue (the timeval in the header is 32bit), collection of jumbo frames and now updated the README.md file of the project and created packages for Debian, Ubuntu, CentOS, OpenSUSE, SLES and I made sure that it can be compiled and use on FreeBSD10 as well.

If you are using or decided not to use this software I would be very happy to hear about it.


January 03, 2016

Harald "LaForge" Welte: Conferences I look forward to in 2016

While I was still active in the Linux kernel development / network security field, I was regularly attending 10 to 15 conferences per year.

Doing so is relatively easy if you earn a decent freelancer salary and are working all by yourself. Running a company funded out of your own pockets, with many issues requiring (or at least benefiting) from personal physical presence in the office changes that.

Nevertheless, after some years of being less of a conference speaker, I'm happy to see that the tide is somewhat changing in 2016.

After my talk at 32C3, I'm looking forward to attending (and sometimes speaking) at events in the first quarter of 2016. Not sure if I can keep up that pace in the following quarters...

FOSDEM

FOSDEM (http://fosdem.org/2016) a classic, and I don't even remember for how many years I've been attending it. I would say it is fair to state it is the single largest event specifically by and for community-oriented free software developers. Feels like home every time.

netdevconf 1.1

netdevconf (http://www.netdevconf.org/1.1/) is actually something I'm really looking forward to. A relatively new grass-roots conference. Deeply technical, and only oriented towards Linux networking hackers. The part of the kernel community that I've known and loved during my old netfilter days.

I'm very happy to attend the event, both for its technical content, and of course to meet old friends like Jozsef, Pablo, etc. I also read that Kuninhiro Ishiguro will be there. I always adored his initial work on Zebra (whose vty code we coincidentally use in almost all osmocom projects as part of libosmovty).

It's great to again see an event that is not driven by commercial / professional conference organizers, high registration fees, and corporate interests. Reminds me of the good old days where Linux was still the underdog and not mainstream... Think of Linuxtag in its early days?

Linaro Connect

I'll be attending Linaro Connect for the first time in many years. It's a pity that one cannot run various open source telecom protocol stack / network element projects and a company and at the same time still be involved deeply in Embedded Linux kernel/system development. So I'll use the opportunity to get some view into that field again - and of course meet old friends.

OsmoDevCon

OsmoDevCon is our annual invitation-only developer meeting of the Osmocom developers. It's very low-profile, basically a no-frills family meeting of the Osmocom community. But really great to meet with all of the team and hearing about their respective experiences / special interest topics.

TelcoSecDay

This (https://www.troopers.de/events/troopers16/580_telcosecday_2016_invitation_only/) is another invitation-only event, organized by the makers of the TROOPERS conference. The idea is to make folks from the classic Telco industry meet with people in IT Security who are looking at Telco related topics. I've been there some years ago, and will finally be able to make it again this year to talk about how the current introduction of 3G/3.5G into the Osmocom network side elements can be used for security research.

Holger "zecke" Freyther: A C++ project without Qt

My primary language is Smalltalk (Pharo and GNU Smalltalk) and I didn't write much C++1x code yet and for a new project I decided to use C++ and experiment. The task was rather simple, receive a message, check if this message type needs to be re-written, start to parse it, make a MongoDB look-up, patch it, send it.

While looking for MongoDB drivers for Qt I only found the pure C++ driver that is working with std::string, exceptions and only offers a blocking interface. At this point I decided to see how painful using stl is these days and opted for a pure stl project. Many years ago the only online documentation came from SGI, there were no examples, no autocompletion in editors, etc...

The application is relatively trivial. It is using lambdas to hook two components (the receiver and the patcher) together, it is using std::thread with lambdas to spawn worker threads because of the blocking interface of the database driver, std::chrono for some ad-hoc StatsD code to add event counters and performance monitoring.

So how did this experiment go?

The C++11/C++14 documentation that is available has improved over the old times, it is still below the quality of the Qt documentation.

Once deploying this on Debian I immediately had odd crashes and I blame ABI issues as my application used C++1x but the mongo-cxx-driver was apparently not using it. I ended up packaging the latest stable ("legacy") standalone driver for debian and made sure it is compiled with C++1x.

The std::chrono API has potential (as it allows me to control the resolution) but the API, specially in C++11 without the new literals, makes easy things difficult. All I wanted was something like QTimer::elapsed() or QElapsedTimer::elapsed(). So I think the Qt API is better designed for this usecase (my measurement resolution is milliseconds and not picoseconds).

For socket code I was using pure C/libc. As my database code was blocking I could make my socket code blocking as well.

I normally use vim but when working with an unknown API I totally enjoy the comfort QtCreator is providing. I was able to jump to the header files and explore the interface. I think it has saved me quite some time while wrangling with the stl.

New languages come with their own build tools these days (Rust/Cargo, Ruby/Rake, ...) and C++1x doesn't have anything to offer here. The options I considered where autotools, CMake or qmake. I think autotools are overkill here, I dislike CMake for various reasons and I like the simplicity of qmake.

In the past I had played with the Google C++ Unittest framework and actually liked it a lot. These days one is asked to copy and paste the framework into the code base. It is an ongoing struggle if one should embed third party libraries or not but as my application is rather small I decided it is a no go. I have used and enjoyed the Qt testlib. So my STL only application is tested using QObject and QCoreApplication.

Conclusion:

The STL (and the documentation) has improved. I still prefer the API design of Qt and when not just interfacing with an library that is using std::string Qt is still my first choice (as it provides event loops, signals/slots, networking, http, etc.).

Holger "zecke" Freyther: OpenCore and Python moving to Github

Some Free Software projects have already moved to Github, some probably plan it and the Python project will move soon. I have not followed the reasons for why the Python project is moving but there is a long list of reasons to move to a platform like github.com. They seem to have a good uptime, offer checkouts through ssh, git, http (good for corporate firewalls) and a subversion interface, they have integrated wiki and ticket management, the fork feature allows an upstream to discover what is being done to the software, the pull requests and the integration with third party providers is great. The last item allows many nice things, specially integrating with a ton of Continuous Integration tools (Travis, Semaphore, Circle, who knows).

Not everything is great though. As a Free Software project one might decide that using proprietary javascript to develop and interact with a Free Software project is not acceptable, one might want to control the repository yourself and then people look for alternatives. At the Osmocom project we are using cgit, mailinglists, patchwork, trac, host our own jenkins and then mirror some of our repositories to github.com for easy access. Another is to find a platform like Github but that is Free and a lot of people look or point to gitlab.com.

From a freedom point of view I think Gitlab is a lot worse than Github. They try to create the illusion that this is a Free Software alternative to Github.com, they offer to host your project but if you want to have the same features for self hosting you will notice that you fell for their marketing. Their website prominently states "Runs GitLab Enterprise" Edition. If you have a look at the feature comparison between the "Community Edition" (the Free Software project) and their open core additions (Enterprise edition) you will notice that many of the extra features are essential.

So when deciding putting your project on github.com or gitlab.com the question is not between proprietary and Free Software but essentially between proprietary and proprietary and as such there is no difference.

December 30, 2015

Harald "LaForge" Welte: 32C3 is over, GSM and GPRS was running fine, osmo-iuh progress

The 32C3 GSM Network

32C3 was great from the Osmocom perspective: We could again run our own cellular network at the event in order to perform load testing with real users. We had 7 BTSs running, each with a single TRX. What was new compared to previous years:

  • OsmoPCU is significantly more robust and stable due to the efforts of Jacob Erlbeck at sysmocom. This means that GPRS is now actually still usable in severe overload situations, like 1000 subscribers sharing only very few kilobits. Of course it will be slow, but at least data still passes through as much as that's possible.
  • We were using half-rate traffic channels from day 2 onwards, in order to enhance capacity. Phones supporting AMR-HR would use that, but then there are lots of old phones that only do classic HR (v1). OsmoNITB with internal MNCC handler supports TCH/H with HR and AMR for at least five years, but the particular combination of OsmoBTS + OsmoNITB + lcr (all master branches) was not yet deployed at previous CCC event networks so far.

Being forced to provide classic HR codec actually revealed several bugs in the existing code:

  • OsmoBTS (at least with the sysmoBTS hardware) is using bit ordering that is not compliant to what the spec says on how GSM-HR frames should be put into RTP frames. We didn't realize this so far, as handing frames from one sysmoBTS to another sysmoBTS of course works, as both use the same (wrong) bit ordering.
  • The ETSI reference implementation of the HR codec has lots of global/static variables, and thus doesn't really support running multiple transcoders in parallel. This is however what lcr was trying (and needing) to do, and it of course failed as state from one transcoder instance was leaking into another. The problem is simple, but the solution not so simple. If you want to avoid re-structuring the entire code in very intrusive ways or running one thread per transcoder instance, then the only solution was to basically memcpy() the entire data section of the transcoding library every time you switch the state from one transcoder instance to the other. It's surprisingly difficult to learn the start + size of that data section at runtime in a portable way, though.

Thanks to our resident voice codec expert Sylvain for debugging and fixing the above two problems.

Thanks also to Daniel and Ulli for taking care of the actual logistics of bringing + installing (+ later unmounting) all associated equipment.

Thanks furthermore to Kevin who has been patiently handling the 'Level 2 Support' cases of people with various problems ending up in the GSM room.

It's great that there is a team taking care of those real-world test networks. We learn a lot more about our software under heavy load situations this way.

osmo-iuh progress + talk

I've been focussing basically full day (and night) over the week ahead of Christmas and during Christmas to bring the osmo-iuh code into a state where we could do a end-to-end demo with a regular phone + hNodeB + osmo-hnbgw + osmo-sgsn + openggsn. Unfortunately I only got it up to the point where we do the PDP CONTEXT ACTIVATION on the signalling plane, with no actual user data going back and forth. And then, for strange reasons, I couldn't even demo that at the end of the talk. Well, in either case, the code has made much progress.

The video of the talk can be found at https://media.ccc.de/v/32c3-7412-running_your_own_3g_3_5g_network#video

meeting friends

The annual CCC congress is always an event where you meet old friends and colleagues. It was great talking to Stefan, Dimitri, Kevin, Nico, Sylvain, Jochen, Sec, Schneider, bunnie and many other hackers. After the event is over, I wish I could continue working together with all those folks the rest of the year, too :/

Some people have been missed dearly. Absence from the CCC congress is not acceptable. You know who you are, if you're reading this ;)

December 28, 2015

Holger "zecke" Freyther: Build or buy a GSM HLR? Is there an alternative?

The classic question in IT is to buy something existing or to build it from scratch. When wanting to buy an off the shelves HLR (that actually works) in most cases the customer will end up in a vendor lock-in:

  • The vendor might enforce to run on a hardware sold by your vendor. This might just be a dell box with a custom front, or really custom hardware in a custom chasis or even requiring you to put an entire rack. Either way you are trapped to a single supplier.
  • It might come with a yearly license (or support fee) and on top of that might be dongled so after a reboot, the service might not start as the new license key has not been copied.
  • The system might not export a configuration interface for what you want. Specially small MVNOs might have specific needs for roaming steering, multi IMSI and you will be sure to pay a premium for these features (even if they are off the shelves extensions).
  • There might be a design flaw in the protocol and you would like to mitigate but the vendor will try to charge a premium from you because the vendor can.



The alternative is to build a component from scratch and the initial progress will be great as the technology is more manageable than many years ago. You will test against the live SS7 network, maybe even encode messages by hand and things appear to work but only then the fun will start. How big is your test suite? Do you have tests for ITU Q787? How will you do load-balancing, database failover? How do you track failures and performance? For many engineering companies this is a bit over their head (one needs to know GSM MAP, need to know ITU SCCP, SIGTRAN, ASN1, TCAP).

But there is a third way and it is available today. Look for a Free Software HLR and give it a try. Check which features are missing and which you want and develop them yourself or ask a company like sysmocom to implement them for you. Once you move the system into production maybe find a support agreement that allows the company to continuously improve the software and responds to you quickly. The benefits for anyone looking for a HLR are obvious:

  • You can run the component on any Linux/FreeBSD system. On physical hardware, on virtualized hardware, together with other services, not with other services. You decide.
  • The software will always be yours. Once you have a running system, there will be nothing (besides time_t overflowing) that has been designed to fail (no license key expires)
  • Independence of a single supplier. You can build a local team to maintain the software, you can find another supplier to maintain it.
  • Built for change. Having access to the source code enables you to modify it, with a Free Software license you are allowed to run your modified versions as well.
The only danger is to make sure to not fall in the OpenCore trap surrounded by many OpenSource projects. Make sure that all you need is available in source and allows you to run modified copies.

December 09, 2015

OpenBSC news feed: Osmocom Berlin Meeting / 2015-12-09

This is the announcement for the re-incarnation of our bi-weekly OsmocomMeeting/Berlin meeting.

Dec 09, 8pm @ CCC Berlin, Marienstr. 11, 10117 Berlin

There is no formal presentation this time, but

  • there will be SIMtrace equipment in case somebody wants to play with it
  • there will be a sysmoBTS with OsmoBTS, OsmoPCU, OsmoNITB, OsmoSGSN and OpenGGSN if somebody wants to play with it

The meeting is open to anyone interested in mobile communications. You do not have to be involved with the Osmocom projects in order to attend. Anyone interested in mobile communications protocols is welcome.

If you are interested to show up, feel free to do so. The meeting is "free as in free beer", despite no actual free beer being around ;)

More information can be found at OsmocomMeeting/Berlin

December 05, 2015

Harald "LaForge" Welte: Volunteer for Openmoko.org USB Product ID maintenance

Back when Openmoko took the fall, we donated the Openmoko, Inc. USB Vendor ID to the community and started the registry of free Product ID allocations at http://wiki.openmoko.org/wiki/USB_Product_IDs

Given my many other involvements and constant overload, I've been doing a poor job at maintaining it, i.e. handling incoming requests.

So I'm looking for somebody who can reliably take care of it, including

  • reviewing if the project fulfills the criteria (hardware or software already released under FOSS license)
  • entering new allocations to the wiki
  • informing applicants of their allocation

The amount of work is actually not that much (like one mail per week), but it needs somebody to reliably respond to the requests in a shorter time frame than I can currently do.

Please let me know if you'd like to volunteer.

Harald "LaForge" Welte: Anyone interested in supporting SMPP interworking at 32C3?

Sylvain brought this up yesterday: Wouldn't it be nice to have some degree of SMS interfacing from OpenBSC/OsmoNITB to the real world at 32C3? It is something that we've never tried so far, and thus definitely worthy of testing.

Of course, full interworking is not possible without assigning public MSISDN to all internal subscribers / 'extensions' how we call them.

But what would most certainly work is to have at least outbound SMS working by means of an external SMPP interface.

The OsmoNITB-internal SMSC speaks SMPP already (in the SMSC role), so we would need to implement some small amount of glue logic that behaves as ESME (external SMS entity) towards both OsmoNITB as well as some public SMS operator/reseller that speaks SMPP again.

Now of course, sending SMS to public operators doesn't come for free. So in case anyone reading this has access to SMPP at public operators, resellers, SMS hubs, it would be interesting to see if there is a chance for some funding/sponsoring of that experiment.

Feel free to contact me if you see a way to make this happen.

December 04, 2015

Harald "LaForge" Welte: python-libsmpp works great with OsmoNITB

Since 2012 we have support for SMPP in OsmoNITB (the network-in-the-box version of OpenBSC). So far I've only used it from C and Erlang code.

Yesterday I gave python-smpplib from https://github.com/podshumok/python-smpplib a try and it worked like a charm. Of course one has to get the details right (like numbering plan indication).

In case anyone is interested in interfacing OsmoNITB SMPP from python, I've put a working example to send SMS at http://cgit.osmocom.org/mncc-python/tree/smpp_test.py

December 01, 2015

Harald "LaForge" Welte: Python tool to talk to OsmoNITB MNCC interface

I've been working on a small python tool that can be used to attach to the MNCC interface of OsmoNITB. It implements the 04.08 CC state machine with our MNCC primitives, including support for RTP bridge mode of the voice streams.

The immediate first use case for this was to be able to generate MT calls to a set of known MSISDNs and load all 14 TCH/H channels of a single-TRX BTS. It will connect the MT calls in pairs, so you end up with 7 MS-to-MS calls.

The first working version of the tool is available from

The code is pretty hacky in some places. That's partially due to the fact that I'm much more familiar in the C, Perl and Erlang world than in python. Still I thought it's a good idea to do it in python to enable more people to use/edit/contribute to it.

I'm happy for review / cleanup suggestion by people with more Python-foo than I have.

Architecturally, I decided to do things a bit erlang-like, where we have finite state machines in an actor models, and message passing between the actors. This is what happens with the GsmCallFsm()'s, which are created by the GsmCallConnector() representing both legs of a call and the MnccActor() that wraps the MNCC socket towards OsmoNITB.

The actual encoding/decoding of MNCC messages is auto-generated from the mncc header file #defines, enums and c-structures by means of ctypes code generation.

mncc_test.py currently drops you into a python shell where you can e.g. start more / new calls by calling functions like connect_call("7839", "3802") from that shell. Exiting the shell by quit() or Ctrl+C will terminate all call FSMs and terminate.

November 29, 2015

Holger "zecke" Freyther: osmo-pcu and a case for Free Software

Last year Jacob and me worked on the osmo-sgsn of OpenBSC. We have improved the stability and reliability of the system and moved it to the next level. By adding the GSUP interface we are able to connect it to our commercial grade Smalltalk MAP stack and use it in the real world production GSM network. While working and manually testing this stack we have not used our osmo-pcu software but another proprietary IP based BTS, after all we didn't want to debug the PCU issues right now.

This year Jacob has taken over as a maintainer of the osmo-pcu, he started with a frequent crash fix (which was introduced due us understanding the specification on TBF re-use better but not the code), he has spent hours and hours reading the specification, studied the log output and has fixed defect after defect and then moved to features. We have tried the software at this years Camp and fixed another round of reliability issues.

Some weeks ago I noticed that the proprietary IP based BTS has been moved from the desk into the shelf. In contrast to the proprietary BTS, issues has a real possibility to be resolved. It might take a long time, it might take one paying another entity to do it but in the end your system will run better. Free Software allows you to genuinely own and use the hardware you have bought!

November 15, 2015

Harald "LaForge" Welte: GSM test network at 32C3, after all

Contrary to my blog post yesterday, it looks like we will have a private GSM network at the CCC congress again, after all.

It appears that Vodafone Germany (who was awarded the former DECT guard band in the 2015 spectrum auctions) is not yet using it in December, and they agreed that we can use it at the 32C3.

With this approval from Vodafone Germany we can now go to the regulator (BNetzA) and obtain the usual test license. Given that we used to get the license in the past, and that Vodafone has agreed, this should be a mere formality.

For the German language readers who appreciate the language of the administration, it will be a Frequenzzuteilung für Versuchszwecke im nichtöffentlichen mobilen Landfunk.

So thanks to Vodafone Germany, who enabled us at least this time to run a network again. By end of 2016 you can be sure they will have put their new spectrum to use, so I'm not that optimistic that this would be possible again.

November 14, 2015

Harald "LaForge" Welte: No GSM test network at 32C3

I currently don't assume that there will be a GSM network at the 32C3.

Ever since OpenBSC was created in 2008, the annual CCC congress was a great opportunity to test OpenBSC and related software with thousands of willing participants. In order to do so, we obtained a test licence from the German regulatory authority. This was never any problem, as there was a chunk of spectrum in the 1800 MHz GSM band that was not allocated to any commercial operator, the so-called DECT guard band. It's called that way as it was kept free in order to ensure there is no interference between 1800 MHz GSM and the neighboring DECT cordless telephones.

Over the decades, it was determined on a EU level that this guard band might not be necessary, or at least not if certain considerations are taken for BTSs deployed in that band.

When the German regulatory authority re-auctioned the GSM spectrum earlier this year, they decided to also auction the frequencies of the former DECT guard band. The DECT guard band was awarded to Vodafone.

This is a pity, as this means that people involved with cellular research or development of cellular technology now have it significantly harder to actually test their systems.

In some other EU member states it is easier, like in the Netherlands or the UK, where the DECT guard band was not treated like any other chunk of the GSM bands, but put under special rules. Not so in Germany.

To make a long story short: Without the explicit permission of any of the commercial mobile operators, it is not possible to run a test/experimental network like we used to ran at the annual CCC congress.

Given that

  • the event is held in the city center (where frequencies are typically used and re-used quite densely), and
  • an operator has nothing to gain from permitting us to test our open source GSM/GPRS implementations,

I think there is little chance that this will become a reality.

If anyone has really good contacts to the radio network planning team of a German mobile operator and wants to prove me wrong: Feel free to contact me by e-mail.

Thanks to everyone involved with the GSM team at the CCC events, particularly Holger Freyther, Daniel Willmann, Stefan Schmidt, Jan Luebbe, Peter Stuge, Sylvain Munaut, Kevin Redon, Andreas Eversberg, Ulli (and everyone else whom I may have forgot, my apologies). It's been a pleasure!

Thanks also to our friends at the POC (Phone Operation Center) who have provided interfacing to the DECT, ISDN, analog and VoIP network at the events. Thanks to roh for helping with our special patch requests. Thanks also to those entities and people who borrowed equipment (like BTSs) in the pre-sysmocom years.

So long, and thanks for all the fish!

November 09, 2015

OpenBSC news feed: Osmocom Berlin User Group / 2015-11-11

This is the announcement for the re-incarnation of our bi-weekly OsmocomMeeting/Berlin meeting.

Nov 11, 8pm @ CCC Berlin, Marienstr. 11, 10117 Berlin

There is no formal presentation this time, but

  • there will be SDR equipment, antenna and a working/tested setup of a gnuradio based MPT1327 decoder
  • there will be SIMtrace equipment in case somebody wants to play with it
  • there will be a sysmoBTS with OsmoBTS, OsmoPCU, OsmoNITB, OsmoSGSN and OpenGGSN if somebody wants to play with it
  • there will be Huawei Femtocells to play with
  • Harald would like to discuss OpenBSC website / documentation improvements

The meeting is open to anyone interested in mobile communications. You do not have to be involved with the Osmocom projects in order to attend. Anyone interested in mobile communications protocols is welcome.

If you are interested to show up, feel free to do so. The meeting is "free as in free beer", despite no actual free beer being around ;)

More information can be found at OsmocomMeeting/Berlin

November 07, 2015

Harald "LaForge" Welte: Progress on the Linux kernel GTP code

It is always sad if you start to develop some project and then never get around finishing it, as there are too many things to take care in parallel. But then, days only have 24 hours...

Back in 2012 I started to write some generic Linux kernel GTP tunneling code. GTP is the GPRS Tunneling Protocol, a protocol between core network elements in GPRS networks, later extended to be used in UMTS and even LTE networks.

GTP is split in a control plane for management and the user plane carrying the actual user IP traffic of a mobile subscriber. So if you're reading this blog via a cellular interent connection, your data is carried in GTP-U within the cellular core network.

To me as a former Linux kernel networking developer, the user plane of GTP (GTP-U) had always belonged into kernel space. It is a tunneling protocol not too different from many other tunneling protocols that already exist (GRE, IPIP, L2TP, PPP, ...) and for the user plane, all it does is basically add a header in one direction and remove the header in the other direction. User data, particularly in networks with many subscribers and/or high bandwidth use.

Also, unlike many other telecom / cellular protocols, GTP is an IP-only protocol with no E1, Frame Relay or ATM legacy. It also has nothing to do with SS7, nor does it use ASN.1 syntax and/or some exotic encoding rules. In summary, it is nothing like any other GSM/3GPP protocol, and looks much more of what you're used from the IETF/Internet world.

Unfortunately I didn't get very far with my code back in 2012, but luckily Pablo Neira (one of my colleagues from netfilter/iptables days) picked it up and brought it along. However, for some time it has been stalled until recently it was thankfully picked up by Andreas Schultz and now receives some attention and discussion, with the clear intention to finish + submit it for mainline inclusion.

The code is now kept in a git repository at http://git.osmocom.org/osmo-gtp-kernel/

Thanks to Pablo and Andreas for picking this up, let's hope this is the last coding sprint before it goes mainline and gets actually used in production.

Harald "LaForge" Welte: Osmocom Berlin meetings

Back in 2012, I started the idea of having a regular, bi-weekly meeting of people interested in mobile communications technology, not only strictly related to the Osmocom projects and software. This was initially called the Osmocom User Group Berlin. The meetings were held twice per month in the rooms of the Chaos Computer Club Berlin.

There are plenty of people that were or still are involved with Osmocom one way or another in Berlin. Think of zecke, alphaone, 2b-as, kevin, nion, max, prom, dexter, myself - just to name a few.

Over the years, I got "too busy" and was no longer able to attend regularly. Some people kept it alive (thanks to dexter!), but eventually they were discontinued in 2013.

Starting in October 2015, I started a revival of the meetings, two have been held already, the third is coming up next week on November 11.

I'm happy that I had the idea of re-starting the meeting. It's good to meet old friends and new people alike. Both times there actually were some new faces around, most of which even had a classic professional telecom background.

In order to emphasize the focus is strictly not on Osmocom alone ( particularly not about its users only), I decided to rename the event to the Osmocom Meeting Berlin.

If you're in Berlin and are interested in mobile communications technology on the protocol and radio side of things, feel free to join us next Wednesday.

November 02, 2015

Harald "LaForge" Welte: Germany's excessive additional requirements for VAT-free intra-EU shipments

Background

At my company sysmocom we are operating a small web-shop providing small tools and accessories for people interested in mobile research. This includes programmable SIM cards, SIM card protocol tracers, adapter cables, duplexers for cellular systems, GPS disciplined clock units, and other things we consider useful to people in and around the various Osmocom projects.

We of course ship domestic, inside the EU and world-wide. And that's where the trouble starts, at least since 2014.

What are VAT-free intra-EU shipments?

As many readers of this blog (at least the European ones) know, inside the EU there is a system by which intra-EU sales between businesses in EU member countries are performed without charging VAT.

This is the result of different countries having different amount of VAT, and the fact that a business can always deduct the VAT it spends on its purchases from the VAT it has to charge on its sales. In order to avoid having to file VAT return statements in each of the countries of your suppliers, the suppliers simply ship their goods without charging VAT in the first place.

In order to have checks and balances, both the supplier and the recipient have to file declarations to their tax authorities, indicating the sales volume and the EU VAT ID of the respective business partners.

So far so good. This concept was reasonably simple to implement and it makes the life easier for all involved businesses, so everyone participates in this scheme.

Of course there always have been some obstacles, particularly here in Germany. For example, you are legally required to confirm the EU-VAT-ID of the buyer before issuing a VAT-free invoice. This confirmation request can be done online

However, the Germany tax authorities invented something unbelievable: A Web-API for confirmation of EU-VAT-IDs that has opening hours. Despite this having rightfully been at the center of ridicule by the German internet community for many years, it still remains in place. So there are certain times of the day where you cannot verify EU-VAT-IDs, and thus cannot sell products VAT-free ;)

But even with that one has gotten used to live.

Gelangensbescheinigung

Now in recent years (since January 1st, 2014) , the German authorities came up with the concept of the Gelangensbescheinigung. To the German reader, this newly invented word already sounds ugly enough. Literal translation is difficult, as it sounds really clumsy. Think of something like a reaching-its-destination-certificate

So now it is no longer sufficient to simply verify the EU-VAT-ID of the buyer, issue the invoice and ship the goods, but you also have to produce such a Gelangensbescheinigung for each and every VAT-free intra-EU shipment. This document needs to include

  • the name and address of the recipient
  • the quantity and designation of the goods sold
  • the place and month when the goods were received
  • the date of when the document was signed
  • the signature of the recipient (not required in case of an e-mail where the e-mail headers show that the messages was transmitted from a server under control of the recipient)

How can you produce such a statement? Well, in the ideal / legal / formal case, you provide a form to your buyer, which he then signs and certifies that he has received the goods in the destination country.

First of all, I find if offensive that I have to ask my customers to make such declarations in the first place. And then even if I accept this and go ahead with it, it is my legal responsibility to ensure that he actually fills this in.

What if the customer doesn't want to fill it in or forgets about it?

Then I as the seller am liable to pay 19% VAT on the purchase he made, despite me never having charged those 19%.

So not only do I have to generate such forms and send them with my goods, but I also need a business process of checking for their return, reminding the customers that their form has not yet been returned, and in the end they can simply not return it and I loose money. Great.

Track+Trace / Courier Services

Now there are some alternate ways in which a Gelangensbescheinigung can be generated. For example by a track+trace protocol of the delivery company. However, the requirements to this track+trace protocol are so high, that at least when I checked in late 2013, the track and trace protocol of UPS did not fulfill the requirements. For example, a track+trace protocol usually doesn't show the quantity and designation of goods. Why would it? UPS just moves a package from A to B, and there is no customs involved that would require to know what's in the package.

Postal Packages

Now let's say you'd like to send your goods by postal service. For low-priced non-urgent goods, that's actually what you generally want to do, as everything else is simply way too expensive compared to the value of the goods.

However, this is only permitted, if the postal service you use produces you with a receipt of having accepted your package, containing the following mandatory information:

  • name and address of the entity issuing the receipt
  • name and address of the sender
  • name and address of the recipient
  • quantity and type of goods
  • date of having receive the goods

Now I don't know how this works in other countries, but in Germany you will not be able to get such a receipt form the postal office.

In fact I inquired several times with the legal department of Deutsche Post, up to the point of sending a registered letter (by Deutsche Post) to Deutsche Post. They have never responded to any of those letters!

So we have the German tax authorities claiming yes, of course you can still do intra-EU shipments to other countries by postal services, you just need to provide a receipt, but then at the same time they ask for a receipt indicating details that no postal receipt would ever show.

Particularly a postal receipt would never confirm what kind of goods you are sending. How would the postal service know? You hand them a package, and they transfer it. It is - rightfully - none of their business what its content may be. So how can you ask them to confirm that certain goods were received for transport ?!?

Summary

So in summary:

Since January 1st, 2014, we now have German tax regulations in force that make VAT free intra-EU shipments extremely difficult to impossible

  • The type of receipt they require from postal services is not provided by Deutsche Post, thereby making it impossible to use Deutsche Post for VAT free intra-EU shipments
  • The type of track+trace protocol issued by UPS does not fulfill the requirements, making it impossible to use them for VAT-free intra-EU shipments
  • The only other option is to get an actual receipt from the customer. If that customer doesn't want to provide this, the German seller is liable to pay the 19% German VAT, despite never having charged that to his customer

Conclusion

To me, the conclusion of all of this can only be one:

German tax authorities do not want German sellers to sell VAT-free goods to businesses in other EU countries. They are actively trying to undermine the VAT principles of the EU. And nobody seem to complain about it or even realize there is a problem.

What a brave new world we live in.

October 31, 2015

Harald "LaForge" Welte: small tools: rtl8168-eeprom

Some time ago I wrote a small Linux command line utility that can be used to (re)program the Ethernet (MAC) address stored in the EEPROM attached to an RTL8168 Ethernet chip.

This is for example useful if you are a system integrator that has its own IEEE OUI range and you would like to put your own MAC address in devices that contain the said Realtek etherent chips (already pre-programmed with some other MAC address).

The source code can be obtaned from: http://git.sysmocom.de/rtl8168-eeprom/

Harald "LaForge" Welte: small tools: gpsdate

In 2013 I wrote a small Linux program that can be usded to set the system clock based on the clock received from a GPS receiver (via gpsd), particularly when a system is first booted. It is similar in purpose to ntpdate, but of course obtains time not from ntp but from the GPS receiver.

This is particularly useful for RTC-less systems without network connectivity, which come up with a completely wrong system clock that needs to be properly set as soon as th GPS receiver finally has acquired a signal.

I asked the ntp hackers if they were interested in merging it into the official code base, and their response was (summarized) that with a then-future release of ntpd this would no longer be needed. So the gpsdate program remains an external utility.

So in case anyone else might find the tool interesting: The source code can be obtained from http://git.sysmocom.de/gpsdate/

October 29, 2015

Harald "LaForge" Welte: Deutsche Bank / unstable interfaces

Deutsche Bank is a large, international bank. They offer services world-wide and are undoubtedly proud of their massive corporate IT department.

Yet, at the same time, they fail to get the most fundamental principles of user/customer-visible interfaces wrong: Don't change them. If you need to change them, manage the change carefully.

In many software projects, keeping the API or other interface stable is paramount. Think of the Linux kernel, where breaking a userspace-visible interface is not permitted. The reasons are simple: If you break that interface, _everyone_ using that interface will need to change their implementation, and will have to synchronize that with the change on the other side of the interface.

The internet online banking system of Deutsche Bank in Germany permits the upload of transactions by their customers in a CSV file format.

And guess what? They change the file format from one day to the other.

  • without informing their users in advance, giving them time to adopt their implementations of that interface
  • without documenting the exact nature of the change
  • adding new fields to the CSV in the middle of the line, rather than at the end of the line, to make sure things break even more

Now if you're running a business and depend on automatizing your payments using the interface provided by Deutsche Bank, this means that you fail to pay your suppliers in time, you hastily drop/delay other (paid!) work that you have to do in order to try to figure out what exactly Deutsche Bank decided to change completely unannounced, from one day to the other.

If at all, I would have expected this from a hobbyist kind of project. But seriously, from one of the worlds' leading banks? An interface that is probably used by thousands and thousands of users? WTF?!?

October 28, 2015

Harald "LaForge" Welte: The VMware GPL case

My absence from blogging meant that I didn't really publicly comment on the continued GPL violations by VMware, and the 2015 legal case that well-known kernel developer Christoph Hellwig has brought forward against VMware.

The most recent update by the Software Freedom Conservancy on the VMware GPL case can be found at https://sfconservancy.org/news/2015/oct/28/vmware-update/

In case anyone ever doubted: I of course join the ranks of the long list of Linux developers and other stakeholders that consider VMware's behavior completely unacceptable, if not outrageous.

For many years they have been linking modified Linux kernel device drivers and entire kernel subsystems into their proprietary vmkernel software (part of ESXi). As an excuse, they have added a thin shim layer under GPLv2 which they call vmklinux. And to make all of this work, they had to add lots of vmklinux specific API to the proprietary vmkernel. All the code runs as one program, in one address space, in the same thread of execution. So basically, it is at the level of the closest possible form of integration between two pieces of code: Function calls within the same thread/process.

In order to make all this work, they had to modify their vmkernel, implement vmklinux and also heavily modify the code they took from Linux in the first place. So the drivers are not usable with mainline linux anymore, and vmklinux is not usable without vmkernel either.

If all the above is not a clear indication that multiple pieces of code form one work/program (and subsequently must be licensed under GNU GPLv2), what should ever be considered that?

To me, it is probably one of the strongest cases one can find about the question of derivative works and the GPL(v2). Of course, all my ramblings have no significance in a court, and the judge may rule based on reports of questionable technical experts. But I'm convinced if the court was well-informed and understood the actual situation here, it would have to rule in favor of Christoph Hellwig and the GPL.

What I really don't get is why VMware puts up the strongest possible defense one can imagine. Not only did they not back down in lengthy out-of-court negotiations with the Software Freedom Conservancy, but also do they defend themselves strongly against the claims in court.

In my many years of doing GPL enforcement, I've rarely seen such a dedication and strong opposition. This shows the true nature of VMware as a malicious, unfair entity that gives a damn sh*t about other peoples' copyright, the Free Software community and its code of conduct as a whole, and the Linux kernel developers in particular.

So let's hope they waste a lot of money in their legal defense, get a sufficient amount of negative PR out of this to the point of tainting their image, and finally obtain a ruling upholding the GPL.

All the best to Christoph and the Conservancy in fighting this fight. For those readers that want to help their cause, I believe they are looking for more supporter donations.

October 27, 2015

Harald "LaForge" Welte: What I've been busy with

Those who don't know me personally and/or stay in touch more closely might be wondering what on earth happened to Harald in the last >= 1 year?

The answer would be long, but I can summarize it to I disappeared into sysmocom. You know, the company that Holger and I founded four years ago, in order to commercially support OpenBSC and related projects, and to build products around it.

In recent years, the team has been growing to the point where in 2015 we had suddenly 9 employees and a handful of freelancers working for us.

But then, that's still a small company, and based on the projects we're involved, that team has to cover a variety of topics (next to the actual GSM/GPRS related work), including

  • mechanical engineering (enclosure design)
  • all types of electrical engineering
    • AC/electrical wiring/fusing on DIN rails
    • AC/DC and isolated DC/DC power supplies (based on modules)
    • digital design
    • analog design
    • RF design
  • prototype manufacturing and testing
  • software development
    • bare-iron bootloader/os/application on Cortex-M0
    • NuttX on Cortex-M3
    • OpenAT applications on Sierra Wireless
    • custom flavors of Linux on several different ARM architectures (TI DaVinci, TI Sitara)
    • drivers for various peripherals including Ethernet Switches, PoE PSE controller
    • lots of system-level software for management, maintenance, control

I've been involved in literally all of those topics, with most of my time spent on the electronics side than on the software side. And if software, the more on the bootloader/RTOS side, than on applications.

So what did we actually build? It's unfortunately still not possible to disclose fully at this point, but it was all related to marine communications technology. GSM being one part of it, but only one of many in the overall picture.

Given the quite challenging breadth/width of the tasks at hand and problem to solve, I'm actually surprised how much we could achieve with such a small team in a limited amount of time. But then, there's virtually no time left, which meant no gpl-violations.org work, no blogging, no progress on the various Osmocom Erlang projects for core network protocols, and last but not least no Taiwan holidays this year.

ately I see light at the end of the tunnel, and there is again a bit ore time to get back to old habits, and thus I

  • resurrected this blog from the dead
  • resurrected various project homepages that have disappeared
  • started some more work on actual telecom stuff (osmo-iuh, for example)
  • restarted the Osmocom Berlin Meeting

October 26, 2015

Harald "LaForge" Welte: Weblog + homepage online again

On October 31st, 2014, I had reeboote my main server for a kernel upgrade, and could not mount the LUKS crypto volume ever again. While the techincal cause for this remains a mystery until today (it has spawned some conspiracy theories), I finally took some time to recover some bits and pieces from elsewhere. I didn't want this situation to drag on for more than a year...

Rather than bringing online the old content using sub-optimal and clumsy tools to generate static content (web sites generated by docbook-xml, blog by blosxom), I decided to give it a fresh start and try nikola, a more modern and actively maintained tool to generate static web pages and blogs.

The blog is now available at http://laforge.gnumonks.org/blog/ (a redirect from the old /weblog is in place, for those who keep broken links for more than 12 months). The RSS feed URLs are different from before, but there are again per-category feeds so people (and planets) can subscribe to the respective category they're interested in.

And yes, I do plan to blog again more regularly, to make this place not just an archive of a decade of blogging, but a place that is alive and thrives with new content.

My personal web site is available at http://laforge.gnumonks.org/ while my (similarly re-vamped) freelancing business web site is also available again at http://hmw-consulting.de/.

I still need to decide what to do about the old http://gnumonks.org/ site. It still has its old manual web 1.0 structure from the late 1990ies.

I've also re-surrected http://openezx.org/ and http://ftp.gpl-devices.org/ as well as http://ftp.gnumonks.org/ (old content). Next in line is gpl-violations.org, which I also intend to convert to nikola for maintenance reasons.

October 19, 2015

OpenBSC news feed: Osmocom Berlin Meeting 2015-10-21

This is the announcement for the re-incarnation of our bi-weekly OsmocomMeeting/Berlin meeting.

Oct 21, 8pm @ CCC Berlin, Marienstr. 11, 10117 Berlin

There is no formal presentation this time, but

  • there will be SDR equipment in case more people are interested to have a look at MPT1327 and/or Tetrapol signals that can be received in Berlin
  • Harald would like to discuss OpenBSC website / documentation improvements

The meeting is open to anyone interested in mobile communications. You do not have to be involved with the Osmocom projects in order to attend. Anyone interested in mobile communications protocols is welcome.

If you are interested to show up, feel free to do so. The meeting is "free as in free beer", despite no actual free beer being around ;)

More information can be found at OsmocomMeeting/Berlin

October 03, 2015

OpenBSC news feed: CANCELLED 2015-10-15 / Berlin: Presentation on OpenAirInterface

THIS EVENT HAS BEEN CANCELLED!

This is an announcement for an "irregular" Berlin Osmocom User Group event.

David Rupprecht of Ruhr-Uni Bochum has offered to give us a presentation sharing his experience in Running OpenAirInterface.

OpenAirInterface (http://openairinterface.eurecom.fr/) is a project of the Eurecom research institute in Sofia Antipoils / France. For many years they have been working towards an open source SDR LTE implementation.

The presentation will be held on

Oct 15, 8pm @ IN-Berlin, Lehrter Str. 53, 10557 Berlin

The meeting is open to anyone interested in mobile communications. You do not have to be involved with the Osmocom projects in order to attend. Anyone interested in mobile communications protocols is welcome.

If you are interested to show up, feel free to do so. The meeting is "free as in free beer", despite no actual free beer being around ;)

More information about the venue can be found at http://www.in-berlin.de/space/

OpenBSC news feed: Osmocom Berlin User Group / 2015-10-07

This is the announcement for the re-incarnation of our bi-weekly OsmoUserGroup/Berlin meeting.

Oct 7, 8pm @ CCC Berlin, Marienstr. 11, 10117 Berlin

Harald will be presenting about the Iuh protocol stack (HNBAP,RUA,RANAP) of UMTS small cells / femtocells and his work towards implementing it as part of Osmocom.

Agenda:

  • 20:00h Welcome
  • 20:15h Presentation about Iuh / osmo-iuh
  • 21:00h Informal meeting / chatting

The meeting is open to anyone interested in mobile communications. You do not have to be involved with the Osmocom projects in order to attend. Anyone interested in mobile communications protocols is welcome.

If you are interested to show up, feel free to do so. The meeting is "free as in free beer", despite no actual free beer being around ;)

More information can be found at OsmoUserGroup/Berlin

June 07, 2015

Holger "zecke" Freyther: Cisco probeless monitoring protocol

The Cisco probeless monitoring protocol (pmp) is a proprietary protocol used by the Cisco ITP. This protocol is used to forward M3UA/MTPL3 messages to another server. The data is being sent on port 33500 using UDP.

Previously I used okteta to study the file format and this time I used Pages and HexFiend. To understand the basic structure one needs to start somewhere. The first assumption for a telco protocol is DER or TLV encoded data. In wireshark one could already see some ascii strings and the first step is to search for a Tag (T) and a Length (L) in front of it. I didn't find a tag but the length was there. At the same time the number of octets to express the length appears to depend on the data that follows. This means the data is certainly not DER encoded. In front of a block of information i found a header that contains the command. The highest bits seems to encode C/R, there is a sequence number and something else I can't decode (doesn't look like a MAC address and not like a time).

I copied the data from HexFiend into an editor document and then used new lines and indentions to illustrate the grouping. This is how I found the "number of messages" field for the data and saw that a message has no size by itself.

After having understood the basic structure I started with a wireshark dissector is around 200 lines of code. It still needs some clean-up, better presentation of the data, checking with fuzzed data packages and then I can propose it for inclusion in wireshark.

February 27, 2015

David Burgess: One Last Word...

Since OpenBTS is a trademark of Range Networks, and since I am no longer at Range Networks, I do not use this blog anymore. I am now at blog.yate.ro.  See you there!

February 13, 2015

OpenBSC news feed: OsmoDevCon from March 27 through March 30, 2015

OsmoDevCon? 2013-03-27 till 2013-03-30, Berlin

Dear fellow Osmcoom developers,

it is my pleasure to finally announce the date + venue of OsmoDevCon2013:

Date: March 27 through March 30, 2015

Place: IN-Berlin, Lehrter Str. 53, Berlin

Like last year, this is an event for developers of the various Osmocom proejects. Reservation and confirmation of reservation is required.

The event is free of charge. The Room is made available by ​IN-Berlin e.V., an Internet related non-profit organization. Lunch catering will be sponsored (so far by sysmocom GmbH, but if any other sponsors come up, we are happy to share the cost).

So all you have to cover is your own travel + accomodation costs, as well as breakfast and dinner. If you are an active developer and cannot afford travel/accomodation, please let me know and I'll see if we can do something about it.

If you would like to attend, please send a message to ​laforge@gnumonks.org applying for registration of the event. The registration deadline is February 20, i.e. one week from now.

There is no detailed schedule of talks yet. I will start a separate discussion suggesting / collecting topics in the next couple of days.

More information is (and will be made) available at ​http://openbsc.osmocom.org/trac/wiki/OsmoDevCon2015

Further discussion regarding the event should be directed at the osmocom-event-orga at lists.osmocom.org mailing list, to avoid cross-posting over the various project-specific lists.

Best regards and happy hacking,

Harald

February 03, 2015

Holger "zecke" Freyther: Moving Jenkins jobs from multi-project to project

When starting to use Jenkins I started with a configuration of multiple nodes and most jobs are multiconfiguration jobs. As it turns out looking at the build results for these jobs is annoying and most jobs don't need to be multiconfiguration jobs at all.

Jenkins doesn't offer to re-configure the jobs and I decided to edit the config.xml file directly. The first thing I did is to change "matrix-project" to "project". The next optional thing is to remove the "axes" nodes and the last and very important bit is to remove the "executionStrategy" node. If the last thing is not done the job will not be parsable and vanish from the jobs list. After making a configuration change I used the reload configuration option to get an immediate result.

December 25, 2014

Holger "zecke" Freyther: The state of mobile telecommunication protocol design and the way ahead

I have been implementing various ETSI/3GPP specifications for more than a decade. At GMIT we provided implementation feedback for DVB-H and OMA BCAST. With the Osmocom project and Sysmocom I have several years of implementing GSM (and UTRAN) specifications on my back.

In general GSM is a great engineering project. It lead to the creation of the ETSI, they adopted the English language for their specifications and they applied the information hiding principle. The group speciale mobile managed to create well described components that communicate through fully specified interfaces. Somebody implementing a SIM card does not need to know about a VLR. Somebody implementing a VLR does not need to know about the AuC. Somebody implementing a BTS doesn't need to know about the MSC.

This summer and in December severe privacy issues on ETSI/3GPP MAP have been revealed. At the 31C3 there will be two in-depth talks about different aspects of it and some of the issues have given us a nice laugh, some the OMG feeling, some gave us pity but I am a software engineer so the question is how did we end up in this situation? ETSI/3GPP MAP was designed at the time SS7 was a walled garden and when it came to telephony one nation could trust another. This trust model fell apart with the liberation of the telephony industry but the specification was not updated. ETSI/3GPP MAP went through several phases and they have had some really bad design choices that are thankfully (or thanks to capitalism) ceased out from the network. All of the issues found and disclosed appear to be bugs in the specification and not a specific implementation. ETSI/3GPP MAP is an old protocol and while one could improve the specification to fix the protocol it is unlikely that new implementations would be rolled out.

But there is hope, there is a new protocol. The protocol was designed in a world where IP was well understood. We had big security issues, viruses, worms, targeted attacks. The basic trust model had changed to a world where one needs to protect oneself. The protocol is called DIAMETER and will power true 4G networks. It is based on the well known RADIUS protocol that many people may know from eduroam.

So do we just need to wait until most networks deploy 4G and we will be more secure against privacy disclosure and other attacks? 3GPP has even created interworking between MAP and DIAMETER. This is done by mapping one or more MAP operations to calls to DIAMETER. Wait what? How can this be possible? It is possible because the fundamental design and trust model is the same. If you tell a HLR/HSS that a subscriber is in your network then they will believe it. While RADIUS was verifying credentials in the home server, in DIAMETER it is not part of what a HSS has to do. This means a subscriber can be still be hijacked.

My understanding of DIAMETER is still very small but it is quite clear that the protocol and mindset is bug-compatible and only the encoding and number of messages has changed.

So where does it leave us? And what should we do?
  • We should have the "public" attend 3GPP meetings to push for better specifications.
  • We should try to stop the DIAMETER roll-out before we have an insecure legacy system long before the old one has been ceased out.
  • We need to push for mobile stacks that only implement L1 and have Free Software that implements the protocol (who wants to have a SIP, UDP and IPsec stack in a proprietary baseband processor?)
  • We need strong End-to-End encryption. But for that we need to be able to control more of the telephony part. Make it possible to run SIP over TLS servers that can interconnect with each other. Make sure that your Network Operator just gives you IP and you handle your telephony yourself.
  • We need to push for protocol design that limits and reduces the overhead around the IP header.
  • We need interest groups and funding that make that possible.


September 06, 2014

Holger "zecke" Freyther: Speeding up my SIP/MGCP Smalltalk Parsers

When creating the MGCP and SIP implementation I didn't want to do string splitting/scanning myself but follow the grammar of the two RFCs. I decided to use the PetitParser framework. PetitParser is a parsing
combinator. Which mostly mean you create small parsers and combine them with things like a sequence (parse this, than that and expect the input to be fully consumed).

For a long time this approach was just okay but I recently started to use the code on a under powered ARM system and the performance started to hurt. The MGCP/SIP Parser is created at runtime and then the result will be used.

Measuring

Besides things being too slow it is important to know how slow it is and to see if a change has made a difference or not. GNU Smalltalk has a Time class that provides a monotonic nanosecond clock. The easiest way to benchmark is to use an approach like:

 [ | start | start := Time nanosecondClock. benchmarkCode. start - Time nanosecondClock] value.

This will give me a number. Now with a language like Smalltalk there will be extra runtime code so there will be a noticeable variance.

Improving the PetitParser GNU Smalltalk port

PetitParser needs to use backtracking to go back in the stream when one element of a choice could not be parsed. This means PetitParser will heavily exercise >>#position, >>#position: and >>#atEnd. In Pharo the implementation has a position variable but in GNU Smalltalk the implementation has a pointer that points to the next character. When a method is called in Smalltalk an activation record (stack frame, context) needs to be created. GNU Smalltalk has the optimization to detect methods that simply return a value or instance variable. In the case of >>#position and >>#position: an arithmetic operation will be executed and can not be optimized. I have decided to change the PetitParser code to use >>#pointer and >>#pointer: to avoid the extra activation cost.

Local PetitParser hacks

In my older port of PetitParser.209 the PPRepeatingParser class will store the parsed elements in an OrderedCollection and at the end convert it to an Array. My client code will simply iterate over the
result and I don't need to pay the price of the extra memory collection and copying.

With Smalltalk and open classes I can simply load my own/improved/reduced version of the >>#parseOn:
and benefit for the speed-gain in my implementation.

Speeding up parser construction

In PetitParser one defines the parser in selectors and on creating everything from the >>#start will be turned into the parser. In case some basic parsers are re-used one can create instance variables and the PetitParser baseclass will then cache the result in that variable. In the past I haven't used this mode too much but I had a lot of parts that are used more than once. This was a significant speed-up in parser construction. At least in PetitParser.209 (I think later versions had some improvements) not every caching will make things more quick. It is good to benchmark both number of created parsers and number construction speed.

Simplifying the Grammar

In both MGCP and SIP I have a SIPGrammar/MGCPGrammar class and then a subclass called SIPParser/MGCPParser that creates a structure my client code can work with. When creating the SIPGrammar/MGCPGrammar I followed the RFC BNF but e.g. for matching the possible parameters I had to create a choice parser with many many choices but all of them are in the form of "Key: Value". So instead of checking the valid keys with the Grammar, I should check the structure in the grammar and do the key validation inside the parser class.

Using PetitParser instead of following the Grammar

In the case of SIP there a lot of rules that specify where whitespace can be. This means I created PPSequenceParsers where some elements consume the space that nobody cares about it. The PPParser class already knows the >>#trim selector to deal with such things. It will automatically take away leading and trailing whitespace.

In SIP (e.g. with the challenge BNF) there is one mandatory parameter followed by optional elements separated by comas. PetitParser has a built-in way to express such things. The >>#separatedBy: selector will take a parser (e.g. one that can parse a comma) as parameter and then parse one or more occurrences.

Using PPObjectPredicateParser

In PetitParser one can write #digit asParser / #word asParser and this will either parse a single digit or a single character. In both cases a PPObjectPredicateParser with a PPCharsetPredicate (with an internal look-up table) will be created. I had many of such occurrences and could simplify the code.

Creating a custom parser

In SIP some parameters can be of the nature of key=value or key="VALUE". The later is a quoted-string that permits certain characters and certain escape sequences. The rule to parse this were nested character parsers of choices of choices. The resulting parser was very slow. A simple string to parse a nonce with a quoted-string could take 40ms. I decided to write my own SIPQuotedStringParser.

Summary

The MGCPParser construction time was in the range of 20 seconds, after the change we are in the ballpark of a second or such and the situation was similar for the SIPParser. A testcase to parse a 401 SIP message went from 200ms to 70ms. This is on a slow ARM with the plain interpreter and there should still be some room for improvements.

Outlook

I think PetitParser could have some further optimizations. Instead of the PPCharSetPredicate and the PPObjectPredicateParser there could be a PPCharSetPredicateParser that avoids the call to >>#value:. and one creating two PPCharSetPredicateParsers and joining them with a PPChoiceParser one could simply join the two look-up tables. This would optimize #digit asParser / #blank asParser and save one LookupTable. The next thing would be to create sparse PPCharSetPredicate when one knows that only a subset will be filled.


In terms of GNU Smalltalk we need to port to GNU lightning 2.0 and we would gain JIT support for ARM and can take it from there as well. Another option would be to start having ByteCode to ByteCode optimizations like inlining often called methods (with and without OSR).

Last but not least we could use MrGwen's GNU lightning bindings and JIT a parser from the PetitParser representation.



June 17, 2014

Holger "zecke" Freyther: Playing with QV4

QtDeclarative is where the fun is. Starting with Qt5.2 a JavaScript Engine written by Digia is used. Compared to JavaScriptCore and V8 this engine is very basic but tightly integrated with the QML and Quick code. Motivated by attending the Qt Developer Summit and my work on GNU Smalltalk I started to look at the VM.

There are some environment variables that help to see what the JavaScript VM is doing and also how it is doing things. The below table gives a quick overview of the available flags and what they do. Specially the usage of QV4_SHOW_IR and QV4_SHOW_ASM helps to understand what is going on.

NameFunction
QV4_NO_SSADo not convert the IR::Function to SSA representation. This disables optimizations as well.
QV4_NO_OPTDo not run the optimizer. This disables dead-code-elimination, constant propagation, copy propagation
QV4_SHOW_IRShow the Intermediate Representation at the various stages of the compilation/optimization.
QV4_SHOW_ASMShow the disassembled code. This requires QtDeclrative to be compiled with CONFIG+=disassembler and without PCH
QV4_NO_REGALLOCDo not use the linear register allocator
QV4_FORCE_INTERPRETERDo not use the JIT but force the interpreter
QV4_MM_AGGRESSIVE_GCRun the GC on every allocation
QV4_MM_STATSPrint the time it took to mark and sweep

March 27, 2014

Holger "zecke" Freyther: Long Time no See (好久不見)

Almost a year ago I took my Qt/DirectFB Jenkins down. With that CI for MIPS/uclibc and DirectFB stopped being done (as far as I know). It was a difficult decision but it reflected my situation. I didn't do a Qt project for a long time and didn't have any Qt related work on the horizon.

My main work involves using Smalltalk (GNU Smalltalk and Pharo) and writing a lot of a C code for the various Osmocom/GSM related projects and the joy of integrating software to build  HW products. I didn't think I would use/write/develop using Qt anytime soon. For sentimental reasons I stayed on the qt-devel mailinglist (I still read it day to day) and followed the work done by Lars, Simon and all the others with great joy.

For an internal project of sysmocom I needed to parse, generate and stream (through HTTP) JSON content. I built a first prototype using GNU Smalltalk and the Iliad Framework. The system was quickly built and we were able to gain some experience with it. Given the amount of data we intended to pipe through the system we wanted to move to a fully compiled version (instead of spending the time tuning the VM). I decided to use the Json support of QtCore, QtNetwork for the HTTP client and planned to use libsoup as the HTTP Server. After some research I stumbled across Tufao and used that one instead and don't regret it. Thanks to QThread and queued signal and slots I was able to make use of more than one CPU core and the application was fun to develop.

I am a Unix Dinosaur so for the buildsystem I shortly considered using qmake but then ended up using autotools. Thanks to the autotroll m4 macros building Qt applications is not that bad. I decided against using cmake as it still feels backward (e.g. no config.log, most scripts don't use pkg-config but some hand rolled magic like the FindBoost thing).

We are building Debian packages using an internal OBS appliance and this way can easily update/deploy our software thanks to that. After deploying every couple of hours the application/thread would crash. The backtrace pointed to QNAM, deleteLater and QObject.. David Faure had documented and fixed various race conditions and after we upgraded our application to use Qt5.2 the crashes stopped occurring. The application was last re-started on the 5th of February and works quite reliable.

This month we started a simple REST server using Qt and Tufao. Maybe I should search for a nice Qt consumer related project too? Qt is definitely here to stay.

December 18, 2013

Holger "zecke" Freyther: Ported DBI and DBD-PostgreSQL from GNU Smalltalk to Pharo

We are building a gsmSCF in Pharo. Normally I would use MongoDB and Voyage to store my objects but as a gsmSCF deals with the balance of users I wanted to use something that supports transactions.

Looking at the SQL support in Pharo. I found SqueakDBX, DBXTalk, native Postgres drivers but all of them lacked prepared statement support and one really doesn't want to format queries by hand.

This is when I decided to port GNU Smalltalks DBI framework and the DBD-PostgreSQL driver. It is not the greatest database framework (QtSql is really close to that) but it does support prepared statements and I have running systems that use this framework.

I have used the gst-convert utility to convert the syntax from GST to FileOut syntax as understood by Pharo and converted the use of DateTime to DateAndTime, Dictionary>>#from: to Dictionary>>#newFrom: and some other incompatibilities. I have fixed some other incompatibilities by hand, e.g. >>#subString: aChar changed to >>#subString: aString and >>#nl to >>#cr. This gave me a port of ROE, DBI and DBD-PostgreSQL relatively quickly. I had some issues with Monticello not properly detecting certain extensions and I had to re-save the methodSource to fix that.

I spent the most time porting the DBD-PostgreSQL FFI code from GNU Smalltalk to NativeBoost of Pharo. The easy part could be converted easily. The literal array in Pharo is interesting and the usage in the >>#nbCallOut: is quite funny. The most difficult part was to create a char** for one call-out and type conversation. I had a de-tour using the NBExternalArrayType and then went to use NativeBoost class>>#allocate: and >>#asNBExternalString. It didn't help that when NativeBoost can't convert a parameter and there will be use-less error message. This is specially annoying when a function takes like 6 arguments or more. Anyway, I had some success yesterday night.

In GNU Smalltalk we are using namespaces so we have DBI.Connection as the entry point. In Pharo this is Connection right now but I should call it DBIConnection in the future. A small example is below and with simple tests it looks like there are no memory leaks in the code. I need to make the code work reliable across image save/restore.

 | connection statement result |
connection := Connection connect: 'dbi:PostgreSQL:dbname=aDb' user: nil password: nil.
statement := connection prepare: 'SELECT * from table WHERE col > $1 AND type = $3'.
result := statement executeWithAll: #(1 'abc').
[result atEnd] whileFalse: [
   | row |
   row := result next.
   row at: 'columnName'.
].

December 17, 2013

OsmocomGMR news feed: Osmo-GMR now implements voice codec decoder

GMR-1 uses an undocumented codec in the AMBE family and as such Osmo-GMR didn't have any implementation of it and converting TCH3 frames to actual audio wasn't possible.

Now, after quite a bit of effort to figure things out, I'm pleased to announce that an open-source decoder is available. It's not perfect and doesn't match the quality of the official decoder but it does produce intelligible voice and will have to do for now.

It's currently available in a separate 'sylvain/codec' branch in the git but should be merged into master soon after a last review of the code.

September 05, 2013

Harald "LaForge" Welte: Problems with OpenVPN on high-latency satellite links

So far I never had a need to look in detail how the OpenVPN protocol actually looks on the wire. It seems like not many people had that much of a close look, as the wireshark plugin is fairly recent (from 2012 I think) while OpenVPN is around for ten more years than that. If I was an OpenVPN developer, the wireshark plugin would be the first thing I'd write to help debugging and development. At least that's what I've been doing from OpenPCD to SIMtrace and through the various GSM and other protocols I encounter...

The reason for my current investigation is some quite strange and yet-unexplained problems when running OpenVPN on high-latency satellite links. I'm not talking about high-bandwidth VSAT or systems with dedicated / guaranteed bandwidth. The links I'm seeing often have RTT (as seen by ICMP echo) of 2 seconds, sometimes even 5. This is of course not only the satellite link, but includes queuing on the ground, possibly the space segment and of course the terminal, including (possibly) access arbitration.

What struck me _very_ odd is that OpenVPN is sending tons of UDP messages with ridiculously small size during the TLS handshake when bringing up the tunnel. Further investigation shows that they actually internally configure a MTU of '0' for the link, which seems to be capped at 100 bytes control payload, plus HMAC and OpenVPN header resulting in 124 to 138 bytes UDP payload.

Now you have to consider that the server certificate (possibly including even a CA certificate) can be quite large, plus all the gazillions of TLS handshaking options in ServerHello, the first message from server to client. This means that OpenVPN transmits that ServerHello in something like 40 to 60 fragments of 100 bytes each! And each of the fragments will have to be acknowledged by the remote end, leading 80 to 120 UDP/IP packets _only_ for the delivery of the TLS ServerHello.

Then you start reviewing the hundreds of OpenVPN configuration options, many of them related to MTU, MSS, fragmentation, etc. There is none for that insanely small default of 100 bytes for control packets during hand-shake. I even read through the related source code, only to find that indeed this behavior seems hard-coded. Some time later I had written a patch to add this option, thanks to Free Software. It seems to work on client and server and brings the ClientHello down to much smaller 4-6 messages.

The fun continues when you see that the timeout for re-transmitting fragments that have not been ACKed yet is 2 seconds. At my satellite RTT times this of course leads to lots of unneeded re-transmissions, simply because the ACK hasn't made its way back to the sender of the original message yet. Luckily there's a configuration option for that.

After the patch and changing that option, the protocol trace looks much more sane. However, I still have problems establishing a tunnel in a number of cases. For some odd reason, the last fragment of the ServerHello is not acknowledged by the client, no matter whether patched or unpatched OpenVPN is being used. I get acknowledgements always only up to fragment N-1 after having transmitted N. That last fragment is then re-transmitted by the server with exponential back-off, and finally some 60 seconds later the server gives up as the TLS handshake didn't finish within that time. Extending the TLS handshake timeout to 120 seconds also doesn't help.

I'm not quite sure why something like 39 out of 39 fragments all get delivered reliably and acknowledged, but always the last fragment (40) doesn't make it to the remote side. That's certainly not random packet loss, but a very deterministic one. Let's see if I can still manage to find out what that might be...

August 04, 2013

Dieter Spaar: Tracing LTE on the phone

As a short additional note to my first experiments running an LTE eNodeB: I wanted to know how this looks on the phone. The protocol I am interested in is RRC (Radio Resource Control) for LTE (specified in TS 36.331). RRC also transports the NAS messages which are exchanged between the eNodeB and the MME.

The USB LTE Dongle I use for my experiments is based on a Qualcomm chipsets. I used Qualcomm based phones for quite a while so I already know some parts of how tracing works on them. I was able to locate the LTE messages in the traces and extend GSMTAP slightly so that it is able to handle LTE RRC messages too.

Here is how it looks in WireShark when a phone registers to the LTE cell:

The red colored lines are uplink messages (from UE to eNodeB), the blue lines are the response from the eNodeB. The trace also contains a few of the System Information messages the phone has to decode to get all the required information to communicate with the eNodeB. Otherwise you can see that the NAS messages exactly match those seen on the S1-AP interface (of course, this is expected).

July 30, 2013

Dieter Spaar: Running your own LTE eNodeB

Finally I got all the required parts together for a fully functional LTE eNodeB. The eNodeB (E-UTRAN Node-B or Evolved Node-B) is the LTE equivalent of a GSM BTS or a 3G Node-B. Considering that its was only less than two years ago when I started to experiment with 3G Node-Bs, obtaining LTE equipment went faster than expected, especially considering that LTE is still in the deployment phase.

Compared to what is needed to run a 3G Node-B, an LTE eNodeB really deserves the "Evolved" in its name. All the functionally of the 3G RNC is contained in the eNodeB (e.g. all the RLC/MAC and radio functionality) and only the higher layers are exposed through the so called "S1" interface. This interface is IP based, there are no such things like E1/T1 or ATM any more.

In case you wonder from where all the configuration parameters for the eNodeB like EARFCN or RF Output Power come from: They have to be configured when setting up the eNodeB, in my case you can configure a few hundred parameters. Luckily most of them have default values which for a start can be taken without modification.

The S1 interface is divided into S1-AP (Application Part) which is specified in TS 36.413. This is the interface to the MME (Mobility Management Entity) which is somehow comparable to a GSM MSC and it uses SCTP. The communication between MME and phone (abbreviated UE like in 3G) are the so called NAS (Non-access stratum) messages, in LTE this is mainly Mobility Management and Session Management, there is no Call Control. LTE NAS is specified in TS 24.301.

The second part of S1 is called S1-U for the user plane traffic which is GTP-U (GPRS Tunneling Protocol) specified in TS 29.281 and using UDP. S1-U connects to the S-GW (Serving Gateway), basically comparable to a GPRS SGSN.

The eNodeB also has a second interface called "X2-AP" (TS 36.423), it is used to interconnect multiple eNodeBs (this is needed for example for handling handover which is done be the eNodeBs themselves). I did not care about X2-AP yet because right now I only have one eNodeB.

So far I wrote a very simple "Proof-of-Concept" so that a phone (in my case an LTE USB Dongle) can register to the LTE cell and transfer data (IP traffic). Just to mention a few of the differences to 3G/GSM:

  • The keys generated by the USIM in the UE (you can use a 3G USIM for LTE too) are not directly used for integrity protection or ciphering. Instead they are used to derive various keys, two of them are the ones used for ciphering/integrity protection on the air interface, this is done in the eNodeB.
  • The NAS messages between UE and MME are also protected (integrity protected and optionally ciphered), another derived set of keys is used for this. This means that even if the keys in the eNodeB can be obtained, the NAS messages are still protected.
  • One Mobility and one Session Management message can be transferred at once in one message over the S1-AP interface, this is for improving speed and saves data transfer on the core network.

Here is how it looks in WireShark when a phone registers to the LTE cell (recent versions of WireShark can handle S1-AP):

The colored lines are uplink messages (from UE to MME), the white lines are the response from the MME. You can see that a default bearer is configured together with attaching to the network. This means that without any further S1-AP messages user plane traffic can start immediately afterwards, at least if the default bearer is sufficient (e.g. the offered quality of service is good enough).

This all is just a very first start with LTE, I hope to find more time for it in the future and explore my eNodeB and LTE further.

June 27, 2013

Holger "zecke" Freyther: Using GNU autotest for running unit tests

This is part of a series of blog posts about testing inside the OpenBSC/Osmocom project. In this post I am focusing on our usage of GNU autotest.

The GNU autoconf ships with a not well known piece of software. It is called GNU autotest and we will focus about it in this blog post.

GNU autotest is a very simple framework/test runner. One needs to define a testsuite and this testsuite will launch test applications and record the exit code, stdout and stderr of the test application. It can diff the output with expected one and fail if it is not matching. Like any of the GNU autotools a log file is kept about the execution of each test. This tool can be nicely integrated with automake's make check and make distcheck. This will execute the testsuite and in case of a test failure fail the build.

The way we use it is also quite simple as well. We create a simple application inside the test/testname directory and most of the time just capture the output on stdout. Currently no unit-testing framework is used, instead a simple application is built that is mostly using OSMO_ASSERT to assert the expectations. In case of a failure the application will abort and print a backtrace. This means that in case of a failure the stdout will not not be as expected and the exit code will be wrong as well and the testcase will be marked as FAILED.

The following will go through the details of enabling autotest in a project.

Enabling GNU autotest

The configure.ac file needs to get a line like this: AC_CONFIG_TESTDIR(tests). It needs to be put after the AC_INIT and AM_INIT_AUTOMAKE directives and make sure AC_OUTPUT lists tests/atlocal


Integrating with the automake

The next thing is to define a testsuite inside the tests/Makefile.am. This is some boilerplate code that creates the testsuite and makes sure it is invoked as part of the build process.


 # The `:;' works around a Bash 3.2 bug when the output is not writeable.  
$(srcdir)/package.m4: $(top_srcdir)/configure.ac
:;{ \
echo '# Signature of the current package.' && \
echo 'm4_define([AT_PACKAGE_NAME],' && \
echo ' [$(PACKAGE_NAME)])' &&; \
echo 'm4_define([AT_PACKAGE_TARNAME],' && \
echo ' [$(PACKAGE_TARNAME)])' && \
echo 'm4_define([AT_PACKAGE_VERSION],' && \
echo ' [$(PACKAGE_VERSION)])' && \
echo 'm4_define([AT_PACKAGE_STRING],' && \
echo ' [$(PACKAGE_STRING)])' && \
echo 'm4_define([AT_PACKAGE_BUGREPORT],' && \
echo ' [$(PACKAGE_BUGREPORT)])'; \
echo 'm4_define([AT_PACKAGE_URL],' && \
echo ' [$(PACKAGE_URL)])'; \
} &>'$(srcdir)/package.m4'
EXTRA_DIST = testsuite.at $(srcdir)/package.m4 $(TESTSUITE)
TESTSUITE = $(srcdir)/testsuite
DISTCLEANFILES = atconfig
check-local: atconfig $(TESTSUITE)
$(SHELL) '$(TESTSUITE)' $(TESTSUITEFLAGS)
installcheck-local: atconfig $(TESTSUITE)
$(SHELL) '$(TESTSUITE)' AUTOTEST_PATH='$(bindir)' \
$(TESTSUITEFLAGS)
clean-local:
test ! -f '$(TESTSUITE)' || \
$(SHELL) '$(TESTSUITE)' --clean
AUTOM4TE = $(SHELL) $(top_srcdir)/missing --run autom4te
AUTOTEST = $(AUTOM4TE) --language=autotest
$(TESTSUITE): $(srcdir)/testsuite.at $(srcdir)/package.m4
$(AUTOTEST) -I '$(srcdir)' -o $@.tmp $@.at
mv $@.tmp $@


Defining a testsuite

The next part is to define which tests will be executed. One needs to create a testsuite.at file with content like the one below:


 AT_INIT  
AT_BANNER([Regression tests.])
AT_SETUP([gsm0408])
AT_KEYWORDS([gsm0408])
cat $abs_srcdir/gsm0408/gsm0408_test.ok > expout
AT_CHECK([$abs_top_builddir/tests/gsm0408/gsm0408_test], [], [expout], [ignore])
AT_CLEANUP

This will initialize the testsuite, create a banner. The lines between AT_SETUP and AT_CLEANUP represent one testcase. In there we are copying the expected output from the source directory into a file called expout and then inside the AT_CHECK directive we specify what to execute and what to do with the output.

Executing a testsuite and dealing with failure

The testsuite will be automatically executed as part of make check and make distcheck. It can also be manually executed by entering the test directory and executing the following.


 $ make testsuite  
make: `testsuite' is up to date.
$ ./testsuite
## ---------------------------------- ##
## openbsc 0.13.0.60-1249 test suite. ##
## ---------------------------------- ##
Regression tests.
1: gsm0408 ok
2: db ok
3: channel ok
4: mgcp ok
5: gprs ok
6: bsc-nat ok
7: bsc-nat-trie ok
8: si ok
9: abis ok
## ------------- ##
## Test results. ##
## ------------- ##
All 9 tests were successful.

In case of a failure the following information will be printed and can be inspected to understand why things went wrong.


  ...  
2: db FAILED (testsuite.at:13)
...
## ------------- ##
## Test results. ##
## ------------- ##
ERROR: All 9 tests were run,
1 failed unexpectedly.
## -------------------------- ##
## testsuite.log was created. ##
## -------------------------- ##
Please send `tests/testsuite.log' and all information you think might help:
To:
Subject: [openbsc 0.13.0.60-1249] testsuite: 2 failed
You may investigate any problem if you feel able to do so, in which
case the test suite provides a good starting point. Its output may
be found below `tests/testsuite.dir'.

You can go to tests/testsuite.dir and have a look at the failing tests. For each failing test there will be one directory that contains a log file about the run and the output of the application. We are using GNU autotest in libosmocore, libosmo-abis, libosmo-sccp, OpenBSC, osmo-bts and cellmgr_ng.

June 06, 2013

Holger "zecke" Freyther: Don't use Hubstaff, if not for the ethical issues, for the security issues

I was recently pointed to a software called Hubstaff. It is meant for virtual companies that do not trust their employees and want to see what their staff is doing. Two central features are to measure the activity and to regularly send screenshots.

Hubstaff does not respect copyright! Their software is installing various GNU GPL and GNU LGPL licensed libraries without respecting the license of these libraries. The sourcecode was not offered at the same place and I didn't see a written offer.

Hubstaff is using HTTP (not encrypted) for all the traffic! The application is sending the login and password in clear text. If you use Hubstaff at the airport, at a local coffee place, in a shared office, everyone can see your password and take over your account. The activities, notes and screenshots are transferred in clear as well. This means that everybody can look at potential confidential information sent from your employee to the Hubstaff server.

To make it worse, it appears trivial that your employees will send you wrong activity information and screenshots. In general this is a game your employees will win but Hubstaff gives a huge head start to your employees.

In short Hubstaff does not respect copyright law, they don't value your data and they have not the slightest clue about security/privacy. No sane business will trust them.


Update: Hubstaff claims that starting from version 0.8.0 SSL is used for the connection to their server, they also claim that their GPL violations have been addressed. I have not verified those claims.

June 05, 2013

Harald "LaForge" Welte: Attending HITCON and COSCUP in Taipei

It is my pleasure to attend the HITCON 2013 and COSCUP 2013 conferences in July/August this year. They are both in Taipei. HITCON is a hacker/security event, while COSCUP is a pure Free/Open Source Software conference.

At both events I will be speaking at the growing list of GSM related tools that are available these days, like OpenBSC, OsmcoomBB, SIMtrace, OsmoSGSN, OsmoBTS, OsmoSDR, etc. As they are both FOSS projects and useful in a security context, this fits well within the scope of both events.

Given that I'm going to be back to Taiwan, I'm looking very much forward to meeting old friends and former colleagues from my Openmoko days in Taipei. God, do I miss those days. While terribly stressful, they still are the most exciting days of my career so far.

And yes, I'm also going to use the opportunity for a continuation of my motorbike riding in this beautiful country.

June 03, 2013

Harald "LaForge" Welte: Rest In Peace, Atul Chitnis

Today, very sad news has reached me: Atul Chitnis has passed away. Most people outside of India will most likely not recognize the name: He has been instrumental in pioneering the BBS community in India, and the founder and leader of the Linux Bangalore and later FOSS.in conferences, held annually in Bangalore.

I myself first met Atul about ten years ago, and had the honor of being invited to speak at many of the conferences he was involved in. Besides that professional connection, we became friends. The warmth and affection with which I was accepted by him and his family during my many trips to Bangalore is without comparison. I was treated and accepted like a family member, despite just being this random free software hacker from Germany who is always way too busy to return the amount of kindness.

Despite the 17 year age difference, there was a connection between the two of us. Not just the mutual respect for each others' work, but something else. It might have been partially due to his German roots. It might have been the similarities in our journey through technology. We both started out in the BBS community with analog modems, we both started to write DOS software in the past, before turning to Linux. We both became heavily involved in mobile technology around the same time: He during his work at Geodesic, I working for Openmoko. Only in recent years his indulgence in Apple products was slightly irritating ;)

Only five weeks ago I had visited Atul. Given the state of his health, it was clear that this might very well be the last time that we meet each other. I'm sad that this now actually turned out to become the thruth. It would have been great to meet again at the end of the year (the typical FOSS.in schedule).

My heartfelt condolences to his family. Particularly to his wonderful wife Shubha, his daughther Anjali, his mother and brother. [who I'm only not calling by their name in this post as they deserve some privacy and their Identities is not listed on Atuls wikipedia page].

Atul was 51 years old. Way too young to die. Yet, he has managed to created a legacy that will extend long beyond his life. He profoundly influenced generations of technology enthusiasts in India and beyond.

May 27, 2013

David Burgess: Return to Pfarrkirchen

Three years ago, we ran an OpenBTS workshop in Pfarrkirchen, hosted by Dieter Spaar at his farm there. Everyone in attendance had a great time, learned a lot about GSM and OpenBTS and had some good opportunities to meet other OpenBTS users from around the world.

We are doing it again at the end of June. For more information follow this link. I hope to see you there.

May 06, 2013

Holger "zecke" Freyther: Spring cleaning of the qt-jenkins.moiji-mobile.com

About a week ago I asked for help/support on the Continuous Integration of Qt for the combination of Linux/MIPS/UCLIBC/DirectFB and due the lack of feedback I have removed the DNS entry and cleaned up my system.

This means that currently there is no (public) build testing for any of DirectFB, MIPS, UCLIBC and the bitrot will increase over time. I have asked on the mailinglist about the removal of the Broadcom 97425 device support as it likely to be unused now.


April 30, 2013

Holger "zecke" Freyther: Interested in MIPS/UCLIBC/DirectFB becoming a Tier1 platform?

Are you running Qt on a MIPS based system? Is your toolchain using UCLIBC? Do plan to use Qt with DirectFB? If not you can probably stop reading.

During the Qt5 development the above was my primary development platform and I spent hours improving the platform and the Qt support. I descended down to the kernel and implemented (and later moved) userspace callchain support for MIPS [1][2] in perf. This allows to get stacktraces/callchains for userspace binaries even when there is no framepointer. I stress-tested the DirectFB platform plugin and found various issues in DirectFB, e.g. this memleak. I modified the V8 MIPS JIT to provide the necessary routines for QML. While doing this I noticed that the ARM implementation is broken and helped to fix it.

At the time Nokia was still using Puls. This meant that getting an external build to integrate with their infrastructure was not possible. So I started to setup a Jenkins for DirectFB and Qt myself. The Qt Jenkins is compiling QtBase, QtJsBackend, QtXmlPatterns, QtDeclarative and QtWebKit for MIPS/Linux/UCLIBC. On top of these there a daily builds for the various QtBase configurations (dist, large, full, medium, small, minimal) and running the V8 unit tests using the built-in simulator for ARM and MIPS. The goal was to extend this to run the all the Qt tests on real hardware. The unit that supported my work was shut-down before I could implement it and the platform work has mostly been in maintenance mode since then.

This has all worked nicely for the release up to Qt 5.0 but when Qt5.1 got merged into the stable branch and received some updates the build started to break and I don't have enough spare time to fix that.

If anyone is interested in either taking over the CI or helping to make this part of my work again I would be very happy.


April 13, 2013

Holger "zecke" Freyther: Migrating *.osmocom.org trac installations to a new host

Yesterday I migrated all trac installations but openbsc.osmocom.org to a new host. We are now running trac version 0.12 and all the used plugins should be installed. As part of the upgrade all tracs should be available via https.

There are various cleanups to do in the next couple of weeks. We should run a similar trac.ini on all the installations, we need to migrate from SQLite to MySQL/MariaDB, all login pages/POSTS should redirect to the https instead of doing a POST/basic auth in plain text.

We are now using a frontend nginx and the /trac/chrome/* are served from a cache and your browser is asked to cache them for 90 hours. This should already reduce the load on the server a bit and should result in better page loads.

April 11, 2013

David Burgess: Range and SS7Ware at CCA

Range Networks and SS7Ware will be presenting their One Core Network solution for mobile networks, based on OpenBTS and Yate, at the Competitive Carriers Association Expo, 17-19 April in New Orleans.

The Open Core Network is the first flexibile, scalable and affordable solution for mobile carriers in rural areas, both for new networks and for expansion of existing networks. Come see us at booth #137. If you would like to meet with them please contact info @ rangenetworks.com or office @ ss7ware.com.


March 31, 2013

Dieter Spaar: Sometimes you need Wireshark to get an RF Power Amplifier running

I recently had the need for an RF Power Amplifier for the UMTS 2100 MHz band. As I didn't have a suitable amplifier at hand, I though of using one of the RF Power Amplifier modules of my Nokia Ultrasite cabinet which is a Node-B configured for UMTS 2100 MHz. The RF Power Amplifier modules in the cabinet can easily be removed, they are directly connected to the power supply line of the cabinet and they have standard RF connectors for RF In and RF Out. So it should be quite easy to operate them outside the cabinet.

Connecting power to the RF Power Amplifier module was easy, however this wasn't enough, the amplifier didn't amplify any RF although the status LED of the module was on. Also the power consumption was quite low, so obviously the actual RF Power Amplifier in the module was not yet on. I didn't want to modify the electronic inside the module so I had to find out how the module gets enabled in the cabinet.

Nearly all of the various modules in the Nokia Ultrasite cabinet are connected at the backplane to Ethernet through a hub (yes, a hub, there is no switch). This Ethernet network is mainly used for controlling the modules, fast data transfer between the modules is done somewhere else. The RF Power Amplifier module is no exception, it's connected to this control Ethernet network too. The configuration and management port of the cabinet is also an Ethernet port connected to the same control network. And because an Ethernet hub is used, all the traffic on this network can be monitored without any further tricks if you are connected to the cabinet's configuration and management port.

So I run Wireshark and captured a trace of the traffic on the cabinet's control network. I already knew the IP address of the RF Power Amplifier module from running it externally, however it only responded to a ping but no TCP or UDP port was open. (BTW, if you wonder how the module gets its IP address: depending on the slot where the module is inserted into the cabinet, different ID pins of the module are grounded. The levels on those ID pins define the MAC and IP address of the module).

From the trace I was able to find out hot it works, it's actually quite evolved:

  • First a certain UDP packet has to be send to the RF Power Amplifier module. Not just destination IP address and port have to match, also the source IP address and port have to be a specific value, if not the module just reports an ICMP error that the port is unreachable (that's the reason why I didn't find an open port during my first try).
  • This UDP package causes the module to initiate a TCP connection back to the sender, when the TCP connection is open the module sends some kind of initialization message and expects an acknowledgment.
  • Only after the TCP connection is open and properly initialized, another UDP package can be send to the module which finally turns the RF Power Amplifier inside the module on.
I am not sure why the TCP connection is necessary, so far I only found out that it is used to control the status LED of the module and to return the operational status (e.g. if a fault was detected). Over UDP it's also possible to query the serial number of the module or to tell it to periodically return RF power measurements. So as you can see, sometimes even just a "simple" thing like an RF Power Amplifier can require quite complicated interaction to run it.

March 29, 2013

OpenBSC news feed: osmocom.org server outage

For about one week from March 20, 2013 on most of the osmocom.org severs and services have been offline due to a hardware failure in the physical machine that was hosting all the virtual machines related to the project.

As I was on extensive business related travel, I couldn't go to the data center and take care of fixing the problem myself. So we had the hard disks sent to my home in Berlin, obtain a machine that has the neccessary SCA-80 connector for the old-fashioned enterprise SCSI hard disks and then start recovery by means of an old backup in the data center + rsync'ing the changes via my slow DSL uplink.

The good news is: Everything is up and running again on different hardware, and there is no data loss as the hard disk was fully intact.

In the coming weeks, we will migrate the various virtual machines over to a new (rented) physical machine, sponsored by sysmocom.

I'd like to thank

  • zecke for providing emergency recovery for the git server
  • roh for providing an old Sun v20z to mount + read the SCA drives
  • noris.net for mailing the hard disks to Berlin
  • the community for their patience with the slow recovery process

Harald "LaForge" Welte: Hardware outage affectiong osmocom.org, deDECTed.org, gpl-violations.org

As usual, murphy's law dictates that problems will occur at the worst possible moment. One of my servers in the data center died on March 20, and it was the machine which hosts the majority of the free software projects that I've created or am involved in. From people.netfilter.org to OpenPCD and OpenEZX to gpl-violations.org and virtually all osmocom.org sites and services.

Recovery was slow as there is no hot spare and none of my other machines in the data center have backplanes for the old SCA-80 hard disks that are in use by that particular machine. So we had to send the disks to Berlin, wait until I'm back there, and then manually rsync everything over to a different box in the data center.

To my big surprise, not many complaints reached me (and yes, my personal and/or business e-mail was not affected in any way)

Recovery is complete now, and I'm looking forward to things getting back to normal soon.

Harald "LaForge" Welte: OsmoDevCon 2013 preparation update

OsmoDevCon 2013 is getting closer every day, and I'm very much looking forward to meet the fellow developers of the various Osmcoom sub-projects. Organization-wise, the catering has now been sorted out, and Holger has managed to get a test license for two ARFCN from the regulatory body without any trouble.

This means that we're more or less all set. The key needs to be picked up from IN-Berlin, and we need to bring some extra extension cords, ethernet switch, power cords and other gear, but that's really only very minor tasks.

There's not as much formal schedule as we used to have last year, which is good as I hope it means we can focus on getting actual work done, as opposed to spending most of the time updating one another about our respective work and progress.

March 02, 2013

Holger "zecke" Freyther: AQBanking with a Deutsche Bank WebSign Card

When I opened an account with the Deutsche Bank I requested a WebSign card. This card has been mostly unused until yesterday when I decided it is time to try it. In theory AQBanking should support this card and everything should work flawlessly but in practice I had to spent several hours in the setup.


Basics

The biggest issue is that most of the available documentation is for older aqbanking versions and I couldn't find a changelog describing how to do the old thing with the new software. So whenever you see a guide using aqhbci-tool you can stop reading as this is for an old version and the commands do not exist in the new one. I understand that the 3rd party documentation is outside of the control of the developer of aqbanking but it would be nice if he could just provide the documentation himself.

I am doing this on Debian Unstable as of the 2.3.2013 and the aqbanking libraries are of version 5.0.24-3 and the libchipcard is version 5.0.3beta-2. I am getting to the exact plugins in a second.

The other part is that the WebSign card is fully configured. There is no requirement for you to download a key into the card or such.

IniLetter

The Deutsche Bank might send you a Ini-Letter, I have done this almost two years ago so I do not remember the details. The AQBanking manual appears to have well described in chapter 6.3.2. I think I followed these instructions back then.

Installation 

The WebSign card is a starcoscard token for AQBanking. To be able to use it you will need to install the libchipcard library. If you only do this you will be greeted with a meaningless error message in the UI asking you to install the libchipcard library. What you actually need are the plugins. In Debian Unstable the package is called libchipcard-libgwenhywfar60-plugins. I have also installed the libchipcard-tools and you should do so too.

The next thing you should do is to check that you have the right card and that you installed everything. I am using a OmniKey card reader and I issued the following command:


$ pcsc_scan
PC/SC device scanner
V 1.4.21 (c) 2001-2011, Ludovic Rousseau
Compiled with PC/SC lite version: 1.8.7
Using reader plug'n play mechanism
Scanning present readers...
0: OMNIKEY AG CardMan 3021 00 00
Sat Mar  2 10:27:23 2013
Reader 0: OMNIKEY AG CardMan 3021 00 00
  Card state: Card inserted,
  ATR: 3B B7 94 00 81 31 FE 65 53 50 4B 32 33 90 00 D1
...
Possibly identified card (using /usr/share/pcsc/smartcard_list.txt):
3B B7 94 00 81 31 FE 65 53 50 4B 32 33 90 00 D1
Giesecke & Devrient Starcos 2.3
Deutsche Bank WebSign (RSA-Card)
G&D StarSign Token


The output shows that the card reader and the card were detected. This means we can continue and check if the libchipcard installation is complete. I am using the gct-tool to show me my user credentials. These include the User-Id and the IP address to use for the Deutsche Bank. I used the following command:

$ gct-tool showuser -t starcoscard
===== Enter Password =====
Please enter the access password for
CARD_ID
You must only enter numbers, not letters.
Input: ENTER_PIN
-------------------------------------------------
Context 1
Service        : BLZ
User Id        : USER_ID
Peer Id        : PEER_ID
Address        : IP
Port           : 3000
System Id      :
Sign Key Id    : A
Verify Key Id  : B
Encipher Key Id: C
Decipher Key Id: D
....

In case you enter the wrong PIN code you have 7 more attempts to enter the right one before the card will be blocked. You will need to use the --forcepin to enter it again. Some other utilities of aqhbci-tool4 appear to become unusable once you have entered the wrong pin. If you do not get the above you are most likely missing the starcoscard plugin.

Configuration

Now that the card is known to work one needs to configure the AQBanking. With the qbankmanager and gnucash I had the issue that no dialogue was presented. So we are going to do this from the console. With the information from above and some knowledge about your bank account (other banking software is capable to take everything from the card) you can use the aqhbci-tool4 to add your user.

$ aqhbci-tool4 adduser -t starcoscard --context=1 -b BLZ  -c ACCOUNT_NR -N YOUR_NAME --hbciversion=300
This will add a new user that will use context #1 of a starcoscard. By default aqhbci-tool4 would select a lower version of HBCI and would use the USER_ID for the customer name. You can verify that the setup is working by importing the accounts and getting the sysid.

$ aqhbci-tool4 getsysid
Locking users
Locking user USER
Executing HBCI jobs
AqHBCI started
Connecting to bank...
Connecting to "IP"
Connected to "IP"
Connected.
There are no tan method descriptions (yet), trying One-Step TAN.
Encoding queue
===== Enter Password =====
Please enter the access password for
CARD_NR
You must only enter numbers, not letters.
Input: ENTER_PIN
Sending queue
Waiting for response
Response received
HBCI: 0010 - Nachricht entgegengenommen. (M)
HBCI: 0020 - Dialogintialisierung erfolgreich. (M)
HBCI: 0020 - Auftrag ausgeführt. (S)
HBCI: 1050 - UPD nicht mehr aktuell. Aktuelle Version folgt. (S)
HBCI: 0020 - Information fehlerfrei entgegengenommen. (S)
Encoding queue
Sending queue
Waiting for response
Response received
HBCI: 0010 - Nachricht entgegengenommen. (M)
HBCI: 0100 - Dialog beendet. (S)
Disconnecting from bank...
Disconnected.
AqHBCI finished.


 If the above fails something is still wrong with your setup. But if it looks like the above you can use the qbankmanager to initiate bank transfers. I hope the above saves someone else the time I had to spent reading the outdated information. In the end it is quite easy to setup.




February 26, 2013

OpenBSC news feed: OsmoDevCon 2013-04-04 till 2013-04-07, Berlin

Dear fellow Osmcoom developers,

it is my pleasure to finally announce the date + venue of OsmoDevCon2013:

Date: April 04 through April 07, 2013

Place: IN-Berlin, Lehrter Str. 53, Berlin

Like last year, this is an event for developers of the various Osmocom proejects. Reservation and confirmation of reservation is required.

The event is free of charge. The Room is made available by IN-Berlin e.V., an Internet related non-profit organization. Lunch catering will be sponsored (so far by sysmocom GmbH, but if any other sponsors come up, we are happy to share the cost).

So all you have to cover is your own travel + accomodation costs, as well as breakfast and dinner. If you are an active developer and cannot afford travel/accomodation, please let me know and I'll see if we can do something about it.

If you would like to attend, please send a message to laforge@gnumonks.org applying for registration of the event. The registration deadline is March 5, i.e. one week from now.

There is no detailed schedule of talks yet. I will start a separate discussion suggesting / collecting topics in the next couple of days.

More information is (and will be made) available at http://openbsc.osmocom.org/trac/wiki/OsmoDevCon2013

Further discussion regarding the event should be directed at the osmocom-event-orga at lists.osmocom.org mailing list, to avoid cross-posting over the various project-specific lists.

Best regards and happy hacking,

Harald

February 19, 2013

OpenBSC news feed: Osmocom Berlin User Group / 2013-02-20

This is the announcement for the latest incarnation of our bi-weekly Osmocom Berlin meeting.

Feb 20, 8pm @ CCC Berlin, Marienstr. 11, 10117 Berlin

There is no formal presentation scheduled for this meeting.

If you are interested to show up, feel free to do so. The meeting is free as in "free beer", despite no actual free beer being around ;)

February 12, 2013

David Burgess: See You in Barcelona



OpenBTS is the only open source implementation of the 3GPP GSM/GPRS and UMTS radio access network (RAN). Yate's SS7 stack is the only open source core network solution certified by Deutsche Telekom's test lab. That's a powerful combination and that's why Range Networks and SS7Ware (the new US office of Null Team) have started a development partnership to provide full cellular networks based on their products.

One exciting result of this US-European collaboration is a unified SIP-based network that supports 2.5G GSM/GPRS, 3G UMTS and 4G LTE RANs all on the same core, simplifying the network architecture and providing a higher return on investment. The Range-SS7Ware open source approach also answers concerns over the security of RAN and core network systems by allowing network operators to inspect the source code of their network software. This combination of open standards and open source gives operators tremendous flexibility in the development of new services and represents a new vision in the deployment of cellular networks.

Range and SS7Ware will be in Barcelona together later this month for the Mobile World Congress. If you would like to meet with them please contact
info @ rangenetworks.com.

February 08, 2013

Harald "LaForge" Welte: Update on what I've been doing

For the better part of a year, this blog has failed to provide you with a lot of updates what I've been doing. This is somewhat relate to a shift from doing freelance work on mainline / FOSS projects like the Linux kernel.

In April 2011, Holger and I started a new company here in Berlin (sysmocom - systems for mobile communications GmbH). This company, among other things, attempts to provide products and services surrounding the various mobile communications related FOSS projects, particularly OpenBSC, OsmoSGSN, OpenGGSN, but also OsmocomBB, and now also OsmoBTS + OsmoPCU, two integral components of our own BTS product called sysmoBTS.

Aside from the usual software development, this entails a variety of other tasks, technical and non-technical. First of all, I did more electrical engineering than I did in the years since Openmoko. And even there, I was only leading the hardware architecture, and didn't actually have to capture schematics or route PCBs myself. So now there are some general-purpose and some customer-specific circuits that had to be done. I really enjoy that work, sometimes even more than software development. Particularly the early/initial design phase can be quite exciting. Selecting components, figuring out how to interconnect them, whether you can fit all of them together in the given amount of GPIOs and other resource of your main CPU, etc. But then even the hand-soldering the first couple of boards is fun, too.

Of all the things I so far had least exposure to is casing and mechanical issues. Luckily we have a contractor working on that for us, but still there are all kinds of issues that can go wrong, where unpopulated PCB footprints can suddenly make contact with a case, or all kinds of issues related to manufacturing tolerances. Another topic is packaging. After all, you want the products to end up in the hands of the customer in a neat, proper and form-fitting package.

On the other hand, there is a lot of administrative work. Sourcing components can sometimes be a PITA, particularly if even distributors like Digikey conspire against you and don't even carry those low quantities of a component that we need for our 100-board low quantity runs. EMC and other measurements for CE approval are a fun topic, too. I've never been involved personally in those, and it has been an interesting venture. Luckily, at least for sysmoBTS, things are looking quite promising now. Customs paperwork, Import/Export related buerocracy (both in Germany as well as other countries) always have new surprises, despite me having experience in dealing with customs for more than 10 years now.

Also significant amount of time is spent on evaluating suppliers and their products, e.g. items like SIM/USIM cards, cavity duplexers, antennas, cables, adapters, power amplifiers and other RF related accessories for our products.

The thing that really caught me off-guard are the German laws on inventory accounting. Basically there is no threshold for low-quantity goods, so as a company on capital (GmbH/AG) you have to account for each and every fscking SMD resistor or capacitor. And then you don't only have to count all those parts, but also put a value at them. Depending on the type of item, you have to use either the purchasing price, or the current market price if you were to buy it again, or the price you expect to sell the item for. Furthermore, the trade law requirements on inventory accounting are different than the tax laws, not often with contradictory aims ;)

In the end it seems the best possible strategy is to put a lot of the low-value inventory into the garbage bin before the end of the financial year, as the value of the product (e.g. 130 SMD resistors in 0402 worth fractions of cents) is so much lower than the cost of counting it. Now that's of course an environmental sin, especially if you consider lots and lots of small and medium-sized companies ending up at that conclusion :(

So all in all, this should give you somewhat of an explanation why there might have been less activity on this blog about exciting technical things. On the one hand, they might relate to customer related projects which are of confidential nature. On the other hand, they might simply be boring things like dealing with transport damage of cavity duplexers from china, or with FedEx billing customs/import fees to the wrong address...

Overall I still have the feeling that I was writing a decent amount of code in 2012 - although there can never be enough :) Most of it was probably either related to OsmoBTS, OpenBSC/OsmoNITB or the various Erlang SS7/TCAP/MAP related projects. The list of more community-oriented projects with long TODO lists is growing, though. I'd like to work on SIMtrace MITM / card emulation support, the CC32RS512 based smartcard OS, libosmosim (there's a first branch in libosmocore.git). Let's hope I can find a bit more time for that kind of stuff this year. You should never give up hope, they say ;)

Copyright (C) 2001-2010 by the respective authors.