Osmocom Planet Osmocom
Open Source Mobile Communications

June 08, 2018

Harald "LaForge" Welte: Re-launching openmoko USB Product ID and Ethernet OUI registry

Some time after Openmoko went out of business, they made available their USB Vendor IDs and IEEE OUI (Ethernet MAC address prefix) available to Open Source Hardware / FOSS projects.

After maintaining that for some years myself, I was unable to find time to continue the work and I had handed it over some time ago to two volunteers. However, as things go, those volunteers also stopped to respond to PID / OUI requests, and we're now launching the third attempt of continuing this service.

As the openmoko.org wiki will soon be moved into an archive of static web pages only, we're also moving the list of allocated PID and OUIs into a git repository.

Since git.openmoko.org is also about to be decommissioned, the repository is now at https://github.com/openmoko/openmoko-usb-oui, next to all the archived openmoko.org repository mirrors.

This also means that in addition to sending an e-mail application for getting an allocation in those ranges, you can now send a pull-request via github.

Thanks to cuvoodoo for volunteering to maintain the Openmoko USB PID and IEEE OUI allocations from now on!

June 06, 2018

Osmocom.org News: Miscellaneous Projects - Osmocom now accepts financial contributions

The Osmocom project (if you count its predecessor OpenBSC) have been running for close to 10 years, creating a large number of Open Source projects related to mobile communications. We have never needed nor wanted any legal entity for it. It's a pure/classic FOSS project, open to contributions from anyone.

Until today, you could only contribute in one of the following forms:

  • by writing code (bug fixes, new features, etc) and submitting it (which means you need to be a developer)
  • by writing documentation / improving the wiki
  • helping other users on the mailing lists, IRC, or in other forums
  • donating cellular equipment (which many don't have)
  • hiring a freelancer or a company to write code and contribute to Osmocom on your behalf
  • buying products or services from companies who dedicate lots of work to Osmocom

However, we've repeatedly getting requests from some individuals who wanted to contribute to the project in an easy way, even if they are not a developer, and/or don't have time, and/or don't have the size of a budget to fund development of entire new features or sub-systems.

Today, Osmocom announces that we have joined Open Collective in order to enable you to make financial contributions, either one-off or recurring.

We'll be using the funds (if we get any!) according to our funding policy outlined at https://opencollective.com/osmocom/expenses/new# in order to pay for expenses such as hosting costs for our servers / IT infrastructure, travel funding for the annual developer conferences, etc. Any and all expenses paid from those funds will be visible on the OpenCollective website. You cannot ask for more transparency than that :)

Thanks in advance for your kind assistance!

May 30, 2018

Osmocom.org News: Osmocom.org Servers - osmocom.org upgraded to redmine 3.4

We've upgraded our redmine project management software from 3.2 to 3.4, which gets us a number of interesting new features and fixes.

This opportunity was also used to install a number of interesting redmine plugins.

Among the intresting new features are:

  • ability to attach tags to issues and wiki pages
  • projects overview page now with multiple columns and no side-pane
  • preview of image attachments at bottom of issues / wiki pages
  • PDF attachment preview in lightbox
  • uploading/attaching of images from clipboard, including browser-based image cropping
  • links to individual users/developers using the "@" notation like "@laforge" leading to laforge
  • ability to add [temporary] banners and/or footers to the site

Let us know if anything was broken during the upgrade. *

May 28, 2018

Dieter Spaar: A personal remark about security research

In the last few years I have done a lot of automotive security research, one of my findings about cars from BMW, which was the result of working on contract for the ADAC, was published in February 2015: the original German version and the translated English version.

A few days ago Tencent Keen Security Lab published their findings about BMW cars in this paper.

While in general I find it great that there is more public research in the area of automotive security, I really don't like it if previous work isn't respected. The report of Tencent Keen Security Lab does not contain any reference to previous work besides their own. I would at least have expected references to previous QNX research (QNX is the OS used in the BMW infotainment ECU) and my work when it comes to the TCB (Telematic Communication Box, the name of the BMW telematic ECU). For example regarding the TCB, the description of the communication protocol with the backend and how encrypted SMS is used, with identical encryption keys in all cars, was already contained in my report. To find out why there are no references to others work I contacted them, but I didn't receive any reply.

Could it be that they really aren't aware of previous work? If you start researching the BMW telematic ECU, you would very soon become aware of NGTP (Next Generation Telematics Protocol), the protocol used to communicate with the BMW backend. Searching on Google for "TCB" and "NGTP" will give you the link to the German article on the first place. OK, it's just the German article and you would have to open it to find the link to the English translation, which requires an additional step.

What if they don't use Google but Baidu? Searching for "TCB" and "NGTP" on Baidu gave this link on the fourth place. I can't read the page, but this seems to be the Chinese translation of the article from Heise, even the text in the diagrams is translated. Interestingly when I tried to reproduce my search from last week on Baidu today, this page no longer appears in the search results, although the page itself still exists.

So for me its looks like this: Tencent Keen Security Lab either simply doesn't give reference to previous work and follows the old Chinese tradition of "Clone and Copy" or they don't have any clue about Information Gathering, which is one the first steps when starting to research a target. In my opinion neither of the two possibilities is what you really want.

May 25, 2018

Osmocom.org News: Osmocom Conferences (OsmoDevCon + OsmoCon) - OsmoCon 2018: Make your Submission Now!

one of the difficulties with OsmoCon2017 last year was that almost nobody submitted talks / discussions within the deadline, early enough to allow for proper planning.

This lead to the situation where the sysmocom team had to come up with a schedule/agenda on their own. Later on much after the CfP deadline,people then squeezed in talks, making the overall schedule too full.

It is up to you to avoid this situation again in 2018 at OsmoCon2018 by submitting your talk RIGHT NOW. We will be very strict regarding late submissions. So if you would like to shape the Agenda of OsmoCon 2018, this is your chance. Please use it.

We will have to create a schedule soon, as [almost] nobody will register to a conference unless the schedule is known. If there's not sufficient contribution in terms of CfP response from the wider community, don't complain later that 90% of the talks are from sysmocom team members and only about the Cellular Network Infrastructure topics.

You have been warned. Please make your CfP submission in time at https://pretalx.sysmocom.de/osmocon2018/cfp before the CfP deadline on 2018-05-30 23:59 (Europe/Berlin)

Thanks for your kind attention.

May 24, 2018

Harald "LaForge" Welte: OsmoCon 2018 CfP closes on 2018-05-30

One of the difficulties with OsmoCon2017 last year was that almost nobody submitted talks / discussions within the deadline, early enough to allow for proper planning.

This lad to the situation where the sysmocom team had to come up with a schedule/agenda on their own. Later on much after the CfP deadline,people then squeezed in talks, making the overall schedule too full.

It is up to you to avoid this situation again in 2018 at OsmoCon2018 by submitting your talk RIGHT NOW. We will be very strict regarding late submissions. So if you would like to shape the Agenda of OsmoCon 2018, this is your chance. Please use it.

We will have to create a schedule soon, as [almost] nobody will register to a conference unless the schedule is known. If there's not sufficient contribution in terms of CfP response from the wider community, don't complain later that 90% of the talks are from sysmocom team members and only about the Cellular Network Infrastructure topics.

You have been warned. Please make your CfP submission in time at https://pretalx.sysmocom.de/osmocon2018/cfp before the CfP deadline on 2018-05-30 23:59 (Europe/Berlin)

Harald "LaForge" Welte: openmoko.org archive down due to datacenter issues

Unfortunately, since about 11:30 am CEST on MAy 24, openmoko.org is down due to some power outage related issues at Hetzner, the hosting company at which openmoko.org has been hosting for more than a decade now.

The problem seems to have caused quite a lot of fall-out tom many servers (Hetzner is hosting some 200k machines, not sure how many affected, though), and Hetzner is anything but verbose when it comes to actually explaining what the issue is.

All they have published is https://www.hetzner-status.de/en.html#8842 - which is rather tight lipped about some power grid issues. But then, what do you have UPSs for if not for "a strong voltage reduction in the local power grid"?

The openmoko.org archive machine is running in Hetzner DC10, by the way. This is where they've had the largest number of tickets.

In any case, we'll have to wait for them to resolve their tickets. They appear to be working day and night on that.

I have a number of machines hosted at Hetzner, and I'm actually rather happy that none of the more important systems were affected that long. Some machines simply lost their uplink connectivity for some minutes, while some others were rebooted (power outage). The openmoko.org archive is the only machine that didn't automatically boot after the outage, maybe the power supply needs replacement.

In any case, I hope the service will be back up again soon.

btw: Guess who's been paying for hosting costs ever since Openmoko, Inc. has shut down? Yes, yours truly. It was OK for something like 9 years, but I want to recursively pull the dynamic content through some cache, which can then be made permanent. The resulting static archive can then be moved to some VM somewhere, without requiring a dedicated root server. That should reduce the costs down to almost nothing.

May 23, 2018

Harald "LaForge" Welte: Mailing List hosting for FOSS Projects

Recently I've encountered several occasions in which a FOSS project would have been interested in some reliable, independent mailing list hosting for their project communication.

I was surprised how difficult it was to find anyone running such a service.

From the user / FOSS project point of view, the criteria that I would have are:

  • operated by some respected entity that is unlikely to turn hostile, discontinue the service or go out of business altogether
  • free of any type of advertisements (we all know how annoying those are)
  • cares about privacy, i.e. doesn't sell the subscriber lists or non-public archives
  • use FOSS to run the service itself, such as GNU mailman, listserv, ezmlm, ...
  • an easy path to migrate away to another service (or self-hosting) as they grow or their requirements change. A simple mail forward to that new address for the related addresses is typically sufficient for that

If you think mailing lists serve no purpose these days anyways, and everyone is on github: Please have a look at the many thousands of FOSS project mailing lists out there still in use. Not everyone wants to introduce a dependency to the whim of a proprietary software-as-a-service provider.

I never had this problem as I always hosted my own mailman instance on lists.gnumonks.org anyway, and all the entities that I've been involved in (whether non-profit or businesses) had their own mailing list hosts. From franken.de in the 1990ies to netfilter.org, openmoko.org and now osmocom.org, we all pride oursevles in self-hosting.

But then there are plenty of smaller projects that neither have the skills nor the funding available. So they go to yahoo groups or some other service that will then hold them hostage without a way to switch their list archives from private to public, without downloadable archives or forwarding in the case they want to move away :(

Of course the larger FOSS projects also have their own list servers, starting from vger.kernel.org to Linux distributions like Debian GNU/Linux. But what if your FOSS project is not specifically Linux related?

The sort-of obvious candidates that I found all don't really fit:

Now don't get me wrong, I'm of course not expecting that there are commercial entities operating free-of charge list hosting services where you neither pay with money, nor your data, nor by becoming a spam receiver.

But still, in the wider context of the Free Software community, I'm seriously surprised that none of the various non-for-profit / non-commercial foundations or associations are offering a public mailing list hosting service for FOSS projects.

One can of course always ask any from the above list and ask for a mailing list even though it's strictly speaking off-topic to them. But who will do that, if he has to ask uninvited for a favor?

I think there's something missing. I don't have the time to set up a related service, but I would certainly want to contribute in terms of funding in case any existing FOSS related legal entity wanted to expand. If you already have a legal entity, abuse contacts, a team of sysadmins, then it's only half the required effort.

May 14, 2018

Osmocom.org News: Osmocom Conferences (OsmoDevCon + OsmoCon) - OsmoCon 2018 CfP deadline extended

The deadline of the Call for Participation (CfP) to the OsmoCon2018 has been extended by two weeks.

This means you still have time until May 30, 2018 to submit your proposal and meet us in October 2018 to talk about your experience, or to share your ideas about Open Source Mobile Communications.

You can find our CfP at https://pretalx.sysmocom.de/osmocon2018/cfp

May 10, 2018

Osmocom.org News: Cellular Network Infrastructure - New Osmocom Cellular software versions released!

The Osmocom project has released new version of the CNI (Cellular Network Infrastructure) software, including OsmoBTS, OsmoBSC, OsmoMGW, OsmoMSC, OsmoHLR, OsmoSGSN, OsmoGGSN.

Thosw new tagged/released versions contain half a year of work since the previous versions released in early November 2017. The primary focus was on bug-fixing and stabilization. Many bugs were introduced during the split of the NITB into individual network elements during 2017, and even more bugs exposed by our ever-growing test coverage, particularly in the Osmocom TTCN-3 test suites.

All-in-all, the post-NITB stack has gained a lot in terms of spec compliance, robustness, stability and features during this period.

You can find pew-compiled binary packages of our latest release for a variety of Debian and Ubuntu GNU/Linux versions at
Latest_Builds.

The developer performing the release related work was Pau Espin. Thanks!

List of tagged versions and link to related ChangeLog

Project Version Changelog
libosmocore 0.11.0 http://git.osmocom.org/libosmocore/plain/debian/changelog?h=0.11.0
libosmo-abis 0.5.0 http://git.osmocom.org/libosmo-abis/plain/debian/changelog?h=0.5.0
libosmo-netif 0.2.0 http://git.osmocom.org/libosmo-netif/plain/debian/changelog?h=0.2.0
OsmoTRX 0.4.0 http://git.osmocom.org/osmo-trx/plain/debian/changelog?h=0.4.0
OsmoBTS 0.8.0 http://git.osmocom.org/osmo-bts/plain/debian/changelog?h=0.8.0
OsmoBSC 1.2.0 http://git.osmocom.org/osmo-bsc/plain/debian/changelog?h=1.2.0
OsmoMSC 1.2.0 http://git.osmocom.org/osmo-msc/plain/debian/changelog?h=1.2.0
OsmoHLR 0.2.1 http://git.osmocom.org/osmo-hlr/plain/debian/changelog?h=0.2.1
osmo-mgw 1.3.0 http://git.osmocom.org/osmo-mgw/plain/debian/changelog?h=1.3.0
osmo-sip-connector 1.1.0 http://git.osmocom.org/osmo-sip-connector/plain/debian/changelog?h=1.1.0
OsmoSTP 0.9.0 http://git.osmocom.org/libosmo-sccp/plain/debian/changelog?h=0.9.0
OsmoSGSN 1.3.0 http://git.osmocom.org/osmo-sgsn/plain/debian/changelog?h=1.3.0
OsmoGGSN 1.2.1 http://git.osmocom.org/osmo-ggsn/plain/debian/changelog?h=1.2.1

Notworthy Changes

Misc

  • GnuTLS fall-back for obtaining randomness
  • support for three-digit MNC throguhout the code-base
  • add talloc introspection via VTY
  • tighter CRTL input parsing
  • stricter VTY config file parsing
  • allow to print only basename of source code file in logging
  • print log level with color-keying of the level name

OsmoTRX

  • OsmoTRX has now a VTY interface and uses Osmocom-style logging + config file
  • use GNU autotest, like other osmocom projects
  • re-introduce support for USRP1 devices
  • build multiple binaries rather than selecting UHD / USRP1 at compile time
  • EFR decoding fixes
  • fix dynamic detection/use of CPU optimization (SSE3 vs SSSE3)
  • various parsing/encoding fixes for trx-control interface
  • add example config file for USRP B200

OsmoBTS

  • higher accuracy reporting of time of arrival
  • fix LAPDm UA memory leak
  • put useful information into RTCP SDES packets
  • fix AMR DTX FSM related crash
  • many fixes related to measurement processing + reporting
  • more robust RSL message parsing + error reporting
  • implement DELETE INDICATION on AGCH overflow
  • fix crashes in IPA DLCX processing
  • fix operation without System Information Type 1

OsmoBSC

  • support all types of Cell Identifier Lists in BSSMAP PAGING
  • fix intra-BSC hand-over (used to work in NITB)
  • fix various error paths in hand-over logic
  • introduce new "handover 2" algorithm from Andreas Eversberg
  • introduce load-based hand-over to balance channel load between overlapping BTSs
  • implement SI2ter + SI2bis rest octets
  • switch to osmo-mgw for handling media/user plane (instead of old osmo-bsc_mgcp)
  • introduce osmo_fsm for subscriber_connection
  • reduce several GSM timers to more reasonable default values (T3113, T3109, T3101, ...)
  • permit codec list with both TCH/F and TCH/H channels
  • permit network supporting more than one A5 cipher
  • fix missing L2 pseudo-length in SI5/SI6 messages
  • introduce Access Control Class (ACC) ramping to deal with overload situations on network power-up
  • switch to "late assignment" by default (we used to do early / very early assignment)
  • many fixes related to 3GPP spec / protocol compliance

OsmoMSC

  • fix various use-after-free in GSUP and CC
  • fix GSM-MILENAGE in presence of 2G keys
  • cancel all paging on IMSI DETACH
  • many fixes related to 3GPP spec / protocol compliance
  • permit network supporting more than one A5 cipher
  • properly pass bearer capabilities between MNCC and CC
  • switch to osmo-mgw for handling media/user plane (instead of old osmo-bsc_mgcp)
  • use dynamic MGCP endpoint allocation using wildcard
  • migrate away from openssl to new libosmocore rand abstraction
  • fixes related to SMS validity time
  • delete expired SMS automatically
  • fix transmission of MM INFO messages
  • fix SMS to non-local subscriber

OsmoHLR

  • fix various crashes
  • fix response to PURGE_MS
  • notify GSUP clients (MSC, SGSN) when HLR subscriber information changes

OsmoMGW (and libosmo-mgcp-client)

  • Introduce osmo_fsm client API
  • various fixes of SDP parser
  • significantly improved compliance with MGCP spec
  • wildcarded endpoint allocation in CRCX

osmo-sip-connector

  • integrate libsofia-sip logging with libosmocore logging
  • add systemd service file

OsmoSTP (and libosmo-sigtran)

  • fix various memory leaks
  • introduce IPA/SCCPlite support (allows translation of SCCPlite to M3UA/SUA)

OsmoSGSN

  • fix some crashes
  • migrate away from openssl to new libosmocore rand abstraction
  • fix display of GTP addresses in VTY

OsmoGGSN (and libgtp)

  • re-introduce support for kernel GTP acceleration (was temporarily removed when migrating from OpenGGSN)
  • fix byte-order of IPCP IPv4 DNS server addresses
  • add support for IPv4v6 End User Addresses
  • Validate packet src addr from MS
  • various sgsnemu fixes

Osmocom.org News: Osmocom.org Servers - expected osmocom.org downtime on Sunday, 2018-05-13

This Sunday (May 13), we're planning to do some of the pending migration of osmocom.org services from our old server to a new machine.

This will cause some downtime. Hopefully it will be minimal, but for sure there will be unexpected events that will mean some of the osmocom.org services will be unavailable at some point during Sunday :/

In terms of priorities, gerrit is high up on the list, so this is the primary focus. If that goes well, redmine is the next on the list.

In case you're interested in the gory details, #3076 is the overview ticket.

May 04, 2018

Osmocom.org News: Osmocom Conferences (OsmoDevCon + OsmoCon) - All OsmoDevCon 2018 Videos Released

All of the video recordings made at the recent 2018 Osmocom Developer Conference (OsmoDevCon2018) have meanwhile been released.

You can find the recordings at https://media.ccc.de/b/conferences/osmocon/osmodevcon18 with a back-up at this youtube playlist

Please excuse the bad audio quality in the Welcome to OsmoDevCon 2018 talk. We didn't pay attention to the clipping and only resolved it towards the end of that talk. All other videos have proper audio quality.

April 29, 2018

Harald "LaForge" Welte: OsmoDevCon 2018 retrospective

One week ago, the annual Osmocom developer meeting (OsmoDevCon 2018) concluded after four long and intense days with old and new friends (schedule can be seen here).

It was already the 7th incarnation of OsmoDevCon, and I have to say that it's really great to see the core Osmocom community come together every year, to share their work and experience with their fellow hackers.

Ever since the beginning we've had the tradition that we look beyond our own projects. In 2012, David Burgess was presenting on OpenBTS. In 2016, Ismael Gomez presented about srsUE + srsLTE, and this year we've had the pleasure of having Sukchan Kim coming all the way from Korea to talk to us about his nextepc project (a FOSS implementation of the Evolved Packet Core, the 4G core network).

What has also been a regular "entertainment" part in recent years are the field trip reports to various [former] satellite/SIGINT/... sites by Dimitri Stolnikov.

All in all, the event has become at least as much about the people than about technology. It's a community of like-minded people that to some part are still working on joint projects, but often work independently and scratch their own itch - whether open source mobile comms related or not.

After some criticism last year, the so-called "unstructured" part of OsmoDevCon has received more time again this year, allowing for exchange among the participants irrespective of any formal / scheduled talk or discussion topic.

In 2018, with the help of c3voc, for the first time ever, we've recorded most of the presentations on video. The results are still in the process of being cut, but are starting to appear at https://media.ccc.de/c/osmodevcon2018.

If you want to join a future OsmoDevCon in person: Make sure you start contributing to any of the many Osmocom member projects now to become eligible. We need you!

Now the sad part is that it will take one entire year until we'll reconvene. May the Osmocom Developer community live long and prosper. I want to meet you guys for many more years at OsmoDevCon!

There is of course the user-oriented OsmoCon 2018 in October, but that's a much larger event with a different audience.

Nevertheless, I'm very much looking forward to that, too.

The OsmoCon 2018 Call for Participation is still running. Please consider submitting talks if you have anything open source mobile communications related to share!

April 28, 2018

Osmocom.org News: OsmoTRX - Osmocom improves USRP1 support in OsmoTRX

The Ettus USRP1 hardware was the first SDR device that could be used to implement a GSM base station. At Osmocom we never removed the related support from OsmoTRX, but it was getting rather hard to use with USRP1 due to two reasons:

  1. libusrp had been removed from gnuradio after gnuradio 3.4.2 in FIXME
  2. OsmoTRX could only be built with either UHD (for USRP2/B2x0/... devices) or with libusrp/USRP1 support

In recent weeks, we have improved the situation by the following measures:

  1. Alexander Huemer took the last released libusrp from gnuradio-3.4.2 and released it at http://git.osmocom.org/libusrp/
  2. Pau Espin packaged this libusrp for Debian, it is now part of our nightly and latest Binary_Packages
  3. Pau Espin re-structured OsmoTRX to build two binaries from one build: osmo-trx-uhd and osmo-trx-usrp

Osmocom.org News: Cellular Network Infrastructure - Osmocom nightly + latest feeds for Ubuntu 18.04

As Ubuntu 18.04 (Bionic Beaver) has been released two days ago, Osmocom has enabled Binary_Packages builds for this new distribution in both nightly and latest. As a result, you can now use our binary package feeds on this most recent incarnation of Ubuntu just like on the other supported distributions.

This was relatively easy due to the openSUSE Build Service (OBS) immediately adding support for Ubuntu 18.04. so all we had to o is enable it, and fix up some minor failures. Thanks to OBS for making supporting new distributions very easy!

April 23, 2018

Osmocom.org News: osmo-fl2k - osmo-fl2k for using USB 3.0 VGA adapters as SDR transmitters

On April 22nd, Osmocom developer Steve Markgraf has released a new Osmocom project called osmo-fl2k during OsmoDevCon2018.

osmo-fl2k allows you to use the DAC contained in Fresco Logic FL2000 USB 3.0 VGA adapters as a SDR transmitter. Rather than rendering video signals to a screen attached to the VGA output, the DAC is used to generate RF signals in base band, possibly relying on the harmonics to achieve signals at much higher frequencies than possible at 165 MS/s in base-band.

The project thus follows the spirit of the earlier rtl-sdr project, which re-uses USB DVB receiver dongles with the RTL2832U chip as SDR receivers. Like-wise, osmo-fl2k creatively re-uses USB 3.0 VGA adaptors as SDR transmitters. The two projects are thus complementary.

Osmocom.org News: Osmocom Conferences (OsmoDevCon + OsmoCon) - OsmoDevCon 2018 concludes

Today, OsmoDevCon2018, the annual Osmocom developer meeting concluded. It featured four days of exciting talks and discussions about open source mobile communications, by developers for developers.

The final schedule containing all 41 individual presentations/discussions/workshops of the four days can be found at https://pretalx.sysmocom.de/osmodevcon2018/schedule/

Thanks to c3voc (the CCC Video team), we have audio+video recordings. Almost all presentations (up to speaker discretion) will be released soon. We will publish another news item once the videos are released.

We'd like to extend our thanks to
  • the 20 speakers (at 25 attendees, that's 80%!)
  • the sponsor sysmocom for organization, covering all lunch + dinner catering as well as some travel funding
  • Lime Microsystems for donating 15 LimeSDR-mini to OsmoDevCon 2018 attendees
  • c3voc for their video recording
  • IN-Berlin for venue hosting (Sat-Mon)
  • CCC Berlin for venue hosting (Fri)

Video recordings of the talks will become available step by step at https://media.ccc.de/b/conferences/osmocon/osmodevcon18

April 22, 2018

Harald "LaForge" Welte: osmo-fl2k - Using USB-VGA dongles as SDR transmitter

Yesterday, during OsmoDevCon 2018, Steve Markgraf released osmo-fl2k, a new Osmocom member project which enables the use of FL2000 USB-VGA adapters as ultra-low-cost SDR transmitters.

How does it work?

A major part of any VGA card has always been a rather fast DAC which generates the 8-bit analog values for (each) red, green and blue at the pixel clock. Given that fast DACs were very rare/expensive (and still are to some extent), the idea of (ab)using the VGA DAC to transmit radio has been followed by many earlier, mostly proof-of-concept projects, such as Tempest for Eliza in 2001.

However, with osmo-fl2k, for the first time it was possible to completely disable the horizontal and vertical blanking, resulting in a continuous stream of pixels (samples). Furthermore, as the supported devices have no frame buffer memory, the samples are streamed directly from host RAM.

As most USB-VGA adapters appear to have no low-pass filters on their DAC outputs, it is possible to use any of the harmonics to transmit signals at much higher frequencies than normally possible within the baseband of the (max) 157 Mega-Samples per seconds that can be achieved.

osmo-fl2k and rtl-sdr

Steve is the creator of the earlier, complementary rtl-sdr software, which since 2012 transforms USB DVB adapters into general-purpose SDR receivers.

Today, six years later, it is hard to think of where SDR would be without rtl-sdr. Reducing the entry cost of SDR receivers nearly down to zero has done a lot for democratization of SDR technology.

There is hence a big chance that his osmo-fl2k project will attain a similar popularity. Having a SDR transmitter for as little as USD 5 is an amazing proposition.

free riders

Please keep in mind that Steve has done rtl-sdr just for fun, to scratch his own itch and for the "hack value". He chose to share his work with the wider public, in source code, under a free software license. He's a very humble person, he doesn't need to stand in the limelight.

Many other people since have built a business around rtl-sdr. They have grabbed domains with his project name, etc. They are now earning money based on what he has done and shared selflessly, without ever contributing back to the pioneering developers who brought this to all of us in the first place.

So, do we want to bet if history repeats itself? How long will it take for vendors showing up online advertising the USB VGA dongles as "SDR transmitter", possibly even with a surcharge? How long will it take for them to include Steve's software without giving proper attribution? How long until they will violate the GNU GPL by not providing the complete corresponding source code to derivative versions they create?

If you want to thank Steve for his amazing work

  • reach out to him personally
  • contribute to his work, e.g.
  • help to maintain it
  • package it for distributions
  • send patches (via osmocom-sdr mailing list)
  • register an osmocom.org account and update the wiki with more information

And last, but not least, carry on the spirit of "hack value" and democratization of software defined radio.

Thank you, Steve! After rtl-sdr and osmo-fl2k, it's hard to guess what will come next :)

April 21, 2018

Kevin Redon: 3GPP specifications revision control

Mobile networks such as GSM, UMTS, and LTE are based on 3GPP specifications; a lot of them (~ 3148). These are the reference documents when trying to understand or implementing any part of the mobile networks. They also evolve with time as details get corrected, features are added, or new radio access technologies such as New Radio (i.e. 5G) are released. Keeping track of the versions and added features is not an easy task, particularly when you are not working on them directly.

April 10, 2018

Osmocom.org News: Osmocom Conferences (OsmoDevCon + OsmoCon) - OsmoCon2018: Save the date + Call for Participation

OsmoCon 2018

OsmoCon (Osmocom Conference) 2018 is the technical conference for Osmocom users, operators and developers!

We are happy to announce the date of OsmoCon 2018. It has been scheduled on October 18 + 19, 2018 and will happen in Berlin, Germany.

For the second time, the Osmocom Conference brings together users, operators and developers of the Osmocom Open Source cellular infrastructure projects, such as OsmoBTS, OsmoBSC, OsmoSGSN, OpenGGSN and others.

Join us for two days of presentations and discussions with the main developers behind Open Source Mobile Communications, as well as commercial and non-profit users of the Osmocom cellular infrastructure software.

You can find some initial information in our wiki at OsmoCon 2018, which will be updated as more information becomes available.

Call for Participation

We're also at the same time announcing the Call for Participation and call on everyone with experiences to share around the Osmocom member projects to submit talks, workshops, discussions or other proposals.

We are particularly looking for contributions about:

  • updates on features/functionality/status of individual Osmocom projects
  • success stories on how Osmocom projects are deployed in practice
  • migration from OsmoNITB to the post-NITB architecture
  • tutorials / workshops on how to setup / analyze Osmocom projects
  • statistics, reporting, operations aspects of Osmocom projects
  • third-party open source utilities to be used with Osmocom projects

April 04, 2018

Osmocom.org News: Osmocom Conferences (OsmoDevCon + OsmoCon) - OsmoDevCon introduces pretalx conference planning

OsmoDevCon has successfully introduced the (obviously free/open source software) pretalx conference planning system for the schedule planning of the upcoming OsmoDevCon2018 conference.

All participants of OsmoDevCon submitting a talk have been receiving e-mails asking to confirm their respective talks/workshops using a confirmation link to the pretalx website.

We are planning to use pretalx for other upcoming events and would like to extend our thanks to the pretalx developers!

March 29, 2018

Harald "LaForge" Welte: udtrace - Unix domain socket tracing

When developing applications that exchange data over sockets, every so often you'd like to analyze exactly what kind of data is exchanged over the socket.

For TCP/UDP/SCTP/DCCP or other IP-based sockets, this is rather easy by means of libpcap and tools like tcpdump, tshark or wireshark. However, for unix domain socket, unfortunately no such general capture/tracing infrastructure exists in the Linux kernel.

Interestingly, even after searching for quite a bit I couldn't find any existing tools for this. This is surprising, as unix domain sockets are used by a variety of programs, from sql servers to bind8 ndc all the way to the systemctl tool to manage systemd.

In absence of any kernel support, the two technologies I can think of to implement this is either systemtap or a LD_PRELOAD wrapper.

However, I couldn't find an example for using either of those two to get traces of unix domain soocket communications.

Ok, so I get to write my own. My first idea hence was to implement something based on top of systemtap, the Linux kernel tracing framework. Unfortunately, systemtap was broken in Debian unstable (which I use for decades) at the time, so I went back to the good old LD_PRELOAD shim library / wrapper approach.

The result is called udtrace and can be found at

git clone git://git.gnumonks.org/udtrace

or alternatively via its github mirror.

Below is a copy+paste of its README file. Let's hope this tool is useful to other developers, too:

udtrace - Unix Domain socket tracing

This is a LD_PRELOAD wrapper library which can be used to trace the data sent and/or received via unix domain sockets.

Unlike IP based communication that can be captured/traced with pcap programs like tcpdump or wireshark, there is no similar mechanism available for unix domain sockets.

This LD_PRELOAD library intercepts the C library function calls of dynamically linked programs. It will detect all file descriptors representing unix domain sockets and will then print traces of all data sent/received via the socket.

Usage

Simply build libudtrace.so using the make command, and then start your to-be-traced program with

LD_PRELOAD=libudtrace.os

e.g.

LD_PRELOAD=libudtrace.so systemctl status

which will produce output like this:

>>> UDTRACE: Unix Domain Socket Trace initialized (TITAN support DISABLED)
>>> UDTRACE: Adding FD 4
>>> UDTRACE: connect(4, "/run/dbus/system_bus_socket")
4 sendmsg W 00415554482045585445524e414c20
4 sendmsg W 3331333033303330
4 sendmsg W 0d0a4e45474f54494154455f554e49585f46440d0a424547494e0d0a
[...]

Output Format

Currently, udtrace will produc the following output:

At time a FD for a unix domain socket is created:

>>> UDTRACE: Adding FD 8

At time a FD for a unix domain socket is closed:

>>> UDTRACE: Removing FD 8

At time a FD for a unix domain socket is bound or connected:

>>> UDTRACE: connect(9, "/tmp/mncc")

When data is read from the socket:

9 read R 00040000050000004403000008000000680000001c0300002c03000000000000

When data is written to the socket:

9 write W 00040000050000004403000008000000680000001c0300002c03000000000000
Where
  • 9 is the file dsecriptor on which the event happened
  • read/write is the name of the syscall, could e.g. also be sendmsg / readv / etc.
  • R|W is Read / Write (from the process point of view)
  • followed by a hex-dump of the raw data. Only data successfully written (or read) will be printed, not the entire buffer passed to the syscall. The rationale is to only print data that was actually sent to or received from the socket.

TITAN decoder support

Getting hex-dumps is nice and fine, but normally one wants to have a more detailed decode of the data that is being passed on the socket.

For TCP based protocols, there is wireshark. But most protocols on unix domain sockets don't follow inter-operable / public standards, so even if one was to pass the traces into wireshark somehow, there would be no decoder.

In the Osmocom project, we already had some type definitions and decoders for our protocols written in the TTCN-3 programming language, using Eclipse TITAN. In order to build those decoders fro MNCC and PCUIF, please use

make ENABLE_TITAN=1

when building the code.

Please note that this introduces a run-time dependency to libttcn3-dynamic.so, which is (at least on Debian GNU/Linux) not installed in a default library search path, so you will have to use something like:

LD_LIBRARY_PATH=/usr/lib/titan LD_PRELOAD=libudtrace.so systemctl status

March 12, 2018

Dieter Spaar: Communication on a FlexRay bus

FlexRay is an automotive bus system which is not as common as the CAN bus, but is used in several car brands, e.g. BMW, Mercedes and Audi. I won't go into the details of FlexRay, there are several good introductions elsewhere.

What is needed to read the data on a FlexRay bus, or even better, actively participate in the communication (send your own data)? Compared to the CAN bus a FlexRay bus is complicated:

  • Sending and receiving on a CAN bus is simple, similar to serial (UART) communication: You basically only need to know the bus speed and you are done.
  • For a FlexRay bus you have to know about 50 more or less important parameters which define things like the length and number of the frames in the static segment or the frames in the dynamic segment.

If you are only interested in seeing the data on the FlexRay bus, than it is actually pretty simple, knowing the communication speed (typically 10 Mbit/s) is enough. There are several oscilloscopes and logic analyzers which can decode the FlexRay protocol. Such a decoder is simple, I use a few lines of AWK script to process the exported CSV file from a cheap logic analyzer to do so (AWK just to show how simple it is).

However if you also want to send data, you need to know nearly all the parameters, otherwise you would most certainly just disturb the bus communication. There are several (expensive) FlexRay analyzers available, which can help to solve this problem.

I wanted to find out if it is also possible to get this done with a relatively cheap (around 100 EUR) development board with a FlexRay interface. While I won't go into the details (maybe this is topic for an upcoming talk), I will just present my experimental setup:

I started with two ECUs (gateway and engine ECU) from a BMW with FlexRay. Those two ECUs are enough for a properly running FlexRay bus (each of those ECUs is a so called coldstart node, you need at least two of them to get the FlexRay bus up and running). To get the development board running with this setup you could start with coarse communication parameters (maybe from measurements with an oscilloscope) and fine-tune them until you can communicate (receive and send frames) without errors. This actually worked quite well.

The next setup were several ECUs from an Audi with FlexRay: I had the gateway, ACC radar and front camera. However only the gateway is a coldstart node, so the FlexRay bus would not start with only those ECUs. I could have bought a second coldstart node ECU (either ABS or airbag), however those ECUs for this specific car are rather expensive. Additionally I wanted to see if it is possible to program the development board as a coldstart node, so I decided to go this way. The problem now is that you don't have a running FlexRay bus to get your first estimation of the communication parameters: the single coldstart node trying to start the bus will only give you a few of them (basically you only have one frame from the static segment). The communication parameters from the BMW won't help also, Audi uses something else (only the 10 Mbit/s bus speed and the 5 ms cycle time are the same). Again I skip the details, but all problems could be resolved and the development board acts as a coldstart node to get the bus running and of course can also properly communicate on the bus.

Lessons learned: You don't necessarily need expensive tools to solve a problem which seems complicated on the first look. If you are willing to spend some time, you can succeed with rather cheap equipment. The additional benefit is that you learn a lot from such an approach.

March 06, 2018

Harald "LaForge" Welte: Report from the Geniatech vs. McHardy GPL violation court hearing

Today, I took some time off to attend the court hearing in the appeal hearing related to a GPL infringement dispute between former netfilter colleague Partrick McHardy and Geniatech Europe

I am not in any way legally involved in the lawsuit on either the plaintiff or the defendant side. However, as a fellow (former) Linux kernel developer myself, and a long-term Free Software community member who strongly believes in the copyleft model, I of course am very interested in this case.

History of the Case

This case is about GPL infringements in consumer electronics devices based on a GNU/Linux operating system, including the Linux kernel and at least some devices netfilter/iptables. The specific devices in question are a series of satellite TV receivers built by a Shenzhen (China) based company Geniatech, which is represented in Europe by Germany-based Geniatech Europe GmbH.

The Geniatech Europe CEO has openly admitted (out of court) that they had some GPL incompliance in the past, and that there was failure on their part that needed to be fixed. However, he was not willing to accept an overly wide claim in the preliminary injunction against his company.

The history of the case is that at some point in July 2017, Patrick McHardy has made a test purchase of a Geniatech Europe product, and found it infringing the GNU General Public License v2. Apparently no source code (and/or written offer) had been provide alongside the binary - a straight-forward violation of the license terms and hence a violation of copyright. The plaintiff then asked the regional court of Cologne to issue a preliminary injunction against the defendant, which was granted on September 8th,2017.

In terms of legal procedure, in Germany, when a plaintiff applies for a preliminary injunction, it is immediately granted by the court after brief review of the filing, without previously hearing the defendant in an oral hearing. If the defendant (like in this case) wishes to appeal the preliminary injunction, it files an appeal which then results in an oral hearing. This is what happened, after which the district court of cologne (Landgericht Koeln) on October 20, 2017 issued ruling 14 O 188/17 partially upholding the injunction.

All in all, nothing particularly unusual about this. There is no dispute about a copyright infringement having existed, and this generally grants any of the copyright holders the right to have the infringing party to cease and desist from any further infringement.

However, this injunction has a very wide scope, stating that the defendant was to cease and desist not only from ever publishing, selling, offering for download any version of Linux (unless being compliant to the license). It furthermore asked the defendant to cease and desist

  • from putting hyperlinks on their website to any version of Linux
  • from asking users to download any version of Linux

unless the conditions of the GPL are met, particularly the clauses related to providing the complete and corresponding source code.

The appeals case at OLG Cologne

The defendant now escalated this to the next higher court, the higher regional court of Cologne (OLG Koeln), asking to withdraw the earlier ruling of the lower court, i.e. removing the injunction with its current scope.

The first very positive surprise at the hearing was the depth in which the OLG court has studied the subject matter of the dispute prior to the hearing. In the many GPL related court cases that I witnessed so far, it was by far the most precise analysis of how Linux kernel development works, and this despite the more than 1000 pages of filings that parties had made to the court to this point.

Just to give you some examples:

  • the court understood that Linux was created by Linus Torvalds in 1991 and released under GPL to facilitate the open and collaborative development
  • the court recognized that there is no co-authorship / joint authorship (German: Miturheber) in the Linux kernel as a whole, as it was not a group of people planning+developing a given program together, but it is a program that has been released by Linus Torvalds and has since been edited by more than 15.000 developers without any "grand joint plan" but rather in successive iterations. This situation constitutes "editing authorship" (German: Bearbeiterurheber)
  • the court further recognized that being listed as "head of the netfilter core team" or a "subsystem maintainer" doesn't necessarily mean that one is contributing copyrightable works. Reviewing thousands of patches doesn't mean you own copyright on them, drawing an analogy to an editorial office at a publisher.
  • the court understood there are plenty of Linux versions that may not even contain any of Patric McHardy's code (such as older versions)

After about 35 minutes of the presiding judge explaining the court's understanding of the case (and how kernel development works), it went on to summarize the summary of their internal elaboration at the court prior to the meeting.

In this summary, the presiding judge stated very clearly that they believe there is some merit to the arguments of the defendant, and that they would be inclined in a ruling favorable to the defendant based on their current understanding of the case.

He cited the following main reasons:

  • The Linux kernel development model does not support the claim of Patrick McHardy having co-authored Linux. In so far, he is only an editing author (Bearbeiterurheber), and not a co-author. Nevertheless, even an editing author has the right to ask for cease and desist, but only on those portions that he authored/edited, and not on the entire Linux kernel.
  • The plaintiff did not sufficiently show what exactly his contributions were and how they were forming themselves copyrightable works
  • The plaintiff did not substantiate what copyrightable contributions he has made outside of netfilter/iptables. His mere listing as general networking subsystem maintainer does not clarify what his copyrightable contributions were
  • The plaintiff being a member of the netfilter core team or even the head of the core team still doesn't support the claim of being a co-author, as netfilter substantially existed since 1999, three years before Patrick's first contribution to netfilter, and five years before joining the core team in 2004.

So all in all, it was clear that the court also thought the ruling on all of Linux was too far-fetching.

The court suggested that it might be better to have regular main proceedings, in which expert witnesses can be called and real evidence has to be provided, as opposed to the constraints of the preliminary procedure that was applied currently.

Some other details that were mentioned somewhere during the hearing:

  • Patrick McHardy apparently unilaterally terminated the license to his works in an e-mail dated 26th of July 2017 towards the defendant. According to the defendant (and general legal opinion, including my own position), this is in turn a violation of the GPLv2, as it only allowed plaintiff to create and publish modified versions of Linux under the obligation that he licenses his works under GPLv2 to any third party, including the defendant. The defendant believes this is abuse of his rights (German: Rechtsmissbraeuchlich).
  • sworn affidavits of senior kernel developer Greg Kroah-Hartman and current netfilter maintainer Pablo Neira were presented in support of some of the defendants claims. The contents of those are unfortunately not public, neither is the contents of the sworn affidavists presented by the plaintiff.
  • The defendant has made substantiated claims in his filings that Patrick McHardy would perform his enforcement activities not with the primary motivation of achieving license compliance, but as a method to generate monetary gain. Such claims include that McHardy has acted in more than 38 cases, in at least one of which he has requested a contractual penalty of 1.8 million EUR. The total amount of monies received as contractual penalties was quoted as over 2 million EUR to this point. Please note that those are claims made by the defendant, which were just reproduced by the court. The court has not assessed their validity. However, the presiding judge explicitly stated that he received a phone calls about this case from a lawyer known to him personally, who supported that large contractual penalties are being paid in other related cases.
  • One argument by the plaintiff seems to center around being listed as a general kernel networking maintainer until 2017 (despite his latest patches being from 2015, and those were netfilter only)

Withdrawal by Patrick McHardy

At some point, the court hearing was temporarily suspended to provide the legal representation of the plaintiff with the opportunity to have a Phone call with the plaintiff to decide if they would want to continue with their request to uphold the preliminary injunction. After a few minutes, the hearing was resumed, with the plaintiff withdrawing their request to uphold the injunction.

As a result, the injunction is now withdrawn, and the plaintiff has to bear all legal costs (court fees, lawyers costs on both sides).

Personal Opinion

For me, this is all of course a difficult topic. With my history of being the first to enforce the GNU GPLv2 in (equally German) court, it is unsurprising that I am in favor of license enforcement being performed by copyright holders.

I believe individual developers who have contributed to the Linux kernel should have the right to enforce the license, if needed. It is important to have distributed copyright, and to avoid a situation where only one (possibly industry friendly) entity would be able to take [legal] action.

I'm not arguing for a "too soft" approach. It's almost 15 years since the first court cases on license violations on (embedded) Linux, and the fact that the problem still exists today clearly shows the industry is very far from having solved a seemingly rather simple problem.

On the other hand, such activities must always be oriented to compliance, and compliance only. Collecting huge amounts of contractual penalties is questionable. And if it was necessary to collect such huge amounts to motivate large corporations to be compliant, then this must be done in the open, with the community knowing about it, and the proceeds of such contractual penalties must be donated to free software related entities to prove that personal financial gain is not a motivation.

The rumors of Patrick performing GPL enforcement for personal financial gain have been around for years. It was initially very hard for me to believe. But as more and more about this became known, and Patrick would refuse to any contact requests by his former netfilter team-mates as well as the wider kernel community make it hard to avoid drawing related conclusions.

We do need enforcement, both out of court and in court. But we need it to happen out of the closet, with the community in the picture, and without financial gain to individuals. The "principles of community oriented enforcement" of the Software Freedom Conservancy as well as the more recent (but much less substantial) kernel enforcement statement represent the most sane and fair approach for how we as a community should deal with license violations.

So am I happy with the outcome? Not entirely. It's good that an over-reaching injunction was removed. But then, a lot of money and effort was wasted on this, without any verdict/ruling. It would have been IMHO better to have a court ruling published, in which the injunction is substantially reduced in scope (e.g. only about netfilter, or specific versions of the kernel, or specific products, not about placing hyperlinks, etc.). It would also have been useful to have some of the other arguments end up in a written ruling of a court, rather than more or less "evaporating" in the spoken word of the hearing today, without advancing legal precedent.

Lessons learned for the developer community

  • In the absence of detailed knowledge on computer programming, legal folks tend to look at "metadata" more, as this is what they can understand.
  • It matters who has which title and when. Should somebody not be an active maintainer, make sure he's not listed as such.
  • If somebody ceases to be a maintainer or developer of a project, remove him or her from the respective lists immediately, not just several years later.
  • Copyright statements do matter. Make sure you don't merge any patches adding copyright statements without being sure they are actually valid.

Lessons learned for the IT industry

  • There may be people doing GPL enforcement for not-so-noble motives
  • Defending yourself against claims in court can very well be worth it, as opposed to simply settling out of court (presumably for some money). The Telefonica case in 2016 <>_ has shown this, as has this current Geniatech case. The legal system can work, if you give it a chance.
  • Nevertheless, if you have violated the license, and one of the copyright holders makes a properly substantiated claim, you still will get injunctions granted against you (and rightfully so). This was just not done in this case (not properly substantiated, scope of injunction too wide/coarse).

Dear Patrick

For years, your former netfilter colleagues and friends wanted to have a conversation with you. You have not returned our invitation so far. Please do reach out to us. We won't bite, but we want to share our views with you, and show you what implications your actions have not only on Linux, but also particularly on the personal and professional lives of the very developers that you worked hand-in-hand with for a decade. It's your decision what you do with that information afterwards, but please do give us a chance to talk. We would greatly appreciate if you'd take up that invitation for such a conversation. Thanks.

January 22, 2018

Dieter Spaar: Modify the VoIP provider of a Speedport ISDN Adapter

The Speedport ISDN Adapter is a relatively cheap VoIP-to-ISDN adapter from Deutsche Telekom. The drawback is that the adapter is "locked" to Deutsche Telekom and the user interface (web interface) is disabled.

Here is how you can access the web interface. While the adapter is already powered on, you have to press both the "Register" and "Reset" button at once for more than 20 seconds. This will temporarily enable a still limited web interface, just point your browser to the IP address of your ISDN adapter. The login password is the "device password" written on the bottom side of the case.

To fully enable the web interface you have to do a bit more. I use "curl" which is available for nearly all operating systems. You have to replace "12345678" with the device password and "192.168.1.2" with the IP address of your adapter.

curl -d "login_pwd=1&pws=12345678" "http://192.168.1.2/cgi-bin/login.cgi"
curl -d "password=12345678&debug_enable=1&uart_tx_eb=1" "http://192.168.1.2/cgi-bin/debug.cgi"

Now the web interface is fully functional and you can modify the settings, e.g. disable the TR-069 interface and enter the parameters for the VoIP account of your provider. "uart_tx_eb" enables the serial console, which offers a few debug commands. However you have to open the case to get access to the serial console.

January 21, 2018

Dieter Spaar: More LTE Base Stations

Since running my first own LTE eNodeB in 2013, I acquired a few more. I now have one from NSN, Ericsson and Huawei. All of them are fully functional including the required Remote Radio Heads to actually transmit.

Operating an LTE eNodeB is not very complicated, these days it is even easier with software like NextEPC. The tricky part is setting up and configuring the LTE eNodeB because there is no standard to do so and every manufacturer has its own way. If you are lucky, you might get an already configured system and there is not much you have to adjust. If your system is new or the configuration has been erased, than it can get complicated, at least for the Ericsson eNodeB I have.

January 19, 2018

Osmocom.org News: Cellular Network Infrastructure - Newspaper article on Osmocom based GSM networks in Mexico

The German newspaper Süddeutsche Zeitung has just published an article about how Rhizomatica provides rural cellular telephony in Oaxaca, Mexico using the Osmocom cellular-infrastructure projects.

You can read the (German) article here

January 01, 2018

Osmocom.org News: Cellular Network Infrastructure - 2017 Osmocom Cellular Infrastructure Review

Osmocom Review 2017

This is a review of the most significant changes and events in the Osmocom Cellular Infrastructure projects in 2017

January 2017

  • announce of first ever public OsmoCon conference in April
  • osmo-bts
    • Add Abis OML failure event reporting
    • fix memory leaks in osmo-bts-{sysmo,lc15} at every channel activation
  • openbsc/osmo-bsc
    • support multiple UARFCNs in SI2quater
  • osmo-hlr
    • add test suite for 2G and 3G authentication
    • fix UMTS AKA re-sync

February 2017

  • weekly manual testing with related weekly test reports to mailing list
  • heads-up about the (lack of a )future of osmo-nitb
  • heads-up about libosmo-sccp SIGTRAN work
  • sysmo-usim-tool
  • libosmo-abis
    • unix domain socket support (for Ericsson L2TP)
  • osmo-bts
    • fix AMR HR DTX FSM logic
    • fix SACCH sending fo system information with enum value > 7
    • osmo-bts-trx: fix RXGAIN and POWER parameters on second TRX
    • fix TCH/H interleaving table bit position
    • sysmoBTS 1020/1100: slow power ramp-up on TRX enable
  • osmo-sgsn
    • fix PDP context activation memory allocation bug
    • integrate support for UMTS AKA
  • openggsn
    • fix kernel-gtp tunnel creation/removal for GTPv1
    • release 0.93

March 2017

  • cgit improvements (about page, change-ID hyperlinks, issue hyperlinks)
  • Add README.md files to all our repositories
  • libosmocore
    • migrate gsm 05.03 coding from OsmoBTS to libosmocore
    • fix SQN / SEQ handling in UMTS AKA
    • 3GPP AoIP message encoding/decoding
  • libosmo-abis
    • fix ever-increasing jitter buffer
  • libosmo-netif
    • handle SCTP in in stream server
    • doxygen documentation on stream an datagram modules
  • osmo-bts
    • octphy: CBCH support
    • include MS timing offset in RSL measurements
  • osmo-sgsn
    • handle IMSIs with leading zeroes
  • osmo-bsc
    • fix T3186 encoding in SI13
    • Improved Ericsson OM2000/RBS2000 support
    • new ctrl2soap proxy in python
  • osmo-hlr
    • add CTRL interface
    • fix SQN/SEQ handling in UMTS AKA

April 2017

  • update of coding style for longer line lengths
  • OsmoCon2017 and OsmoDevCon2017
  • libosmocore
  • libosmo-netif
    • fix file descriptor leak in error paths
    • work around linux kenrel SCTP bug with sender_dry_events
    • RTP marker bit support
  • libosmo-sccp
    • Add new [[libosmo-sigtran:]] library with SS7 AS/ASP Link/Linkset handling, M3UA support, new FSM based SCCP implementation
    • Add osmo-stp program
  • osmo-bts
    • inform BSC of PCU disconnect
    • fix measurement reporting period
    • exclude idle channels from uplink measurement processing
    • octphy: measurement reports

May 2017

  • libosmocore
    • fix embedded builds
    • import and generalise 'sercomm' from osmocom-bb into libosmocore
    • SSE optimized convolutional coder
    • fix wrong GSM FR codec SID frame generation
    • doxygen docs for libosmocoding
  • osmo-bsc
    • TS 04.14 mobile station side loop control
  • osmo-bts
    • consistently check all RSL and OML TLVs for minimum length value
    • fix bit-order in every HR codec parameter (spec compliance)
    • OML get/set attribute handling
    • SI2quater support
    • bypass radio link timeout for lab testing
  • osmo-bsc
    • PCU socket support for BSC-colocated PCU for Ericsson RBS2000
    • reelase 1.0.1
  • M3UA and SUA testing as part of jenkins
  • osmo-gsm-tester produces successful runs with NITB as well as new AoIP

June 2017

  • libosmocore
    • doxygen autobrief
    • doxygen documentation for libosmogb
  • osmo-bts
    • use CLOCK_MONOTONIC timer for GSM frame timer
    • PDTCH loopback support

July 2017

  • Plan for openbsc.git split and code review
  • libosmocore
    • PDP charging characteristics in GSUP
    • PRBS sequence generators
    • multicast IP related helper functions
    • 'make release' target
  • libosmo-sccp
    • SCCP address book
  • osmo-bts
    • new virtual BTS osmo-bts-virtual for testing without radio hardware
    • don't send dummy UI frames on unused BCCH slots on TC=5
    • GSMTAP: don't log/send fill frames consisting of only padding
  • osmo-hlr
    • change to default GSUP port 4222

August 2017

  • Support for SMPP Delivery Receipt / GSM03.40 Status Report
  • Jenkins now executing M3UA, SUA and GGSN testsuite
  • libosmocore
    • fix crash in lapd_est_req()
  • libosmo-abis
    • release 0.4.0
  • osmo-bts
    • osmo-bts-trx: fix MS power control loop
    • release 0.6.0
    • support sending/removing SI13 to/from PCU
  • osmo-bsc
    • indicate R99+ MSC in SI3 to enable UMTS AKA over GERAN
  • osmo-sgsn
    • properly report GERAN/UTRAN mode in PDP CTX ACT REQ to GGSN
  • osmo-msc
    • implement IuCS support
    • split openbsc.git into osmo-bsc.git, osmo-msc.git and osmo-sgsn.git
  • openggsn
    • Add IPv6 address pool and IPV6 user (inner) plane support
    • release 0.94

September 2017

  • libosmocore.git
  • osmo-hlr
    • CTRL interface tests
  • openggsn
    • various cleanups and conversion to osmocom style/apis
    • fork osmo-ggsn from openggsn; obsolete openggsn
  • osmo-ggsn
    • release 1.0.0
    • allow enable/disable of G-PDU sequence numbers on ggsn and sgsnemu
    • sgsnemu: Add IPv6 PDP context support

October 2017

  • new 'osmocom:latest' package feed
  • libosmocore.git
  • libosmo-netif.git
    • release 0.1.0 and 0.1.1
  • libosmo-sccp.git
    • release 0.8.0 and 0.8.1
  • osmo-bts
    • don't require gsm_data_shared.[ch] from openbsc.git anymore
    • fix multiple subsequent SI1quater BCCH FILLING from BSC
    • fix AMR DTX FSM name to avoid invalid characters in name/identifier
    • release 0.7.0
  • osmo-bsc
    • release 1.1.0, 1.1.1 and 1.1.2
  • osmo-hlr
    • replace/reimplement CTRL interface commands
    • release 0.1.0
  • osmo-sgsn
    • release 1.2.0
  • osmo-msc
    • release 1.1.0, 1.1.1 and 1.1.2
    • ensure we default to enable TMSI allocation
  • osmo-ggsn
    • release 1.1.0

November 2017

  • LimeSDR support in osmocom package feeds
  • Add SPDX License identifiers in library projects
  • libosmocore
    • fix SSE3 optimization on non-SSSE3 machines
    • fix memory leaks in various unit tests for memory leak ebugging
    • add counter group introspection via CTRL
    • release 0.10.2
  • libosmo-netif
    • fix another file descriptor leak in stream server implementation
  • libosmo-sccp.git
  • osmo-bts
    • don't abort if oversized RTP packets are received
  • osmo-bsc
    • migrate from osmo-bsc_mgcp to osmo-mgw for switching user plane
    • more SI2quater fixes
    • report per-BTS connection state and uptime in VTY + CTRL
  • osmo-ggsn
    • various improvements in kernel GTP support

December 2017

  • libosmocore
    • improvements of XML export for VTY command reference generation
  • osmo-bts
    • put useful information in RTCP SDES
  • osmo-bsc
    • move lots of counters / KPIs from BSC level down to per-BTS granularity
    • reduce T3101 default from 10s to 3s
    • generate mandatory SI2bis and SI2ter rest octets (on RSL; Um was always fine)
    • reduce T3113 from 60s to 10s
    • various new per-BTS counters
  • osmo-msc
    • SMS database related fixes
    • properly set permitted ciphering algorithms in BSSMAP Cipher Mode Command
    • fix GSM-Milenage in presence of 2G keys
  • osmo-ggsn
    • fix byte order in IPCP IPV4 DNS server addresses

Osmocom.org News: Cellular Network Infrastructure - Osmocom operates GSM/UMTS network at 34C3

A team of Osmocom volunteers has continued the tradition of operating an experimental test network at the annual Chaos Communication Congress 34C3 held in Leipzig (Germany) from December 27 through December 30, 2017.

You can find some more information about this network at the GSM page in the 34C3 wiki and the report by tsaitgaist as part of the 34C3 infrastructure review (from minute 35 onwards of http://live.dus.c3voc.de/relive//34c3/8911/muxed.mp4)

Osmocom thanks
  • Deutsche Telekom for providing us with 3 ARFCN in the 1800 MHz band
  • Bundesnetzagentur for providing the experimental license for UMTS in the 850 MHz band
  • The team of volunteers working on the setup and operation of the network

December 31, 2017

Harald "LaForge" Welte: Osmocom Review 2017

As 2017 has just concluded, let's have a look at the major events and improvements in the Osmocom Cellular Infrastructure projects (i.e. those projects dealing with building protocol stacks and network elements for mobile network infrastructure.

I've prepared a detailed year 2017 summary at the osmocom.org website, but let me write a bit about the most note-worthy topics here.

NITB Split

Once upon a time, we implemented everything needed to operate a GSM network inside a single process called OsmoNITB. Those days are now gone, and we have separate OsmoBSC, OsmoMSC, OsmoHLR, OsmoSTP processes, which use interfaces that are interoperable with non-Osmocom implementations (which is what some of our users require).

This change is certainly the most significant change in the close-to-10-year history of the project. However, we have tried to make it as non-intrusive as possible, by using default point codes and IP addresses which will make the individual processes magically talk to each other if installed on a single machine.

We've also released a OsmoNITB Migration Guide, as well as our usual set of user manuals in order to help our users.

We'll continue to improve the user experience, to re-introduce some of the features lost in the split, such as the ability to attach names to the subscribers.

Testing

We have osmo-gsm-tester together with the two physical setups at the sysmocom office, which continuously run the latest Osmocom components and test an entire matrix of different BTSs, software configurations and modems. However, this tests at super low load, and it tests only signalling so far, not user plane yet. Hence, coverage is limited.

We also have unit tests as part of the 'make check' process, Jenkins based build verification before merging any patches, as well as integration tests for some of the network elements in TTCN-3. This is much more than we had until 2016, but still by far not enough, as we had just seen at the fall-out from the sub-optimal 34C3 event network.

OsmoCon

2017 also marks the year where we've for the first time organized a user-oriented event. It was a huge success, and we will for sure have another OsmoCon incarnation in 2018 (most likely in May or June). It will not be back-to-back with the developer conference OsmoDevCon this time.

SIGTRAN stack

We have a new SIGTRAN stakc with SUA, M3UA and SCCP as well as OsmoSTP. This has been lacking a long time.

OsmoGGSN

We have converted OpenGGSN into a true member of the Osmocom family, thereby deprecating OpenGGSN which we had earlier adopted and maintained.

Harald "LaForge" Welte: 34C3 and its Osmocom GSM/UMTS network

At the 34th annual Chaos Communication Congress, a team of Osmocom folks continued the many years old tradition of operating an experimental Osmocom based GSM network at the event. Though I've originally started that tradition, I'm not involved in installation and/or operation of that network, all the credits go to Lynxis, neels, tsaitgaist and the larger team of volunteers surrounding them. My involvement was only to answer the occasional technical question and to look at bugs that show up in the software during operation, and if possible fix them on-site.

34C3 marks two significant changes in terms of its cellular network:

  • the new post-nitb Osmocom stack was used, with OsmoBSC, OsmoMSC and OsmoHLR
  • both an GSM/GPRS network (on 1800 MHz) was operated ,as well as (for the first time) an UMTS network (in the 850 MHz band)

The good news is: The team did great work building this network from scratch, in a new venue, and without relying on people that have significant experience in network operation. Definitely, the team was considerably larger and more distributed than at the time when I was still running that network.

The bad news is: There was a seemingly endless number of bugs that were discovered while operating this network. Some shortcomings were known before, but the extent and number of bugs uncovered all across the stack was quite devastating to me. Sure, at some point from day 2 onwards we had a network that provided [some level of] service, and as far as I've heard, some ~ 23k calls were switched over it. But that was after more than two days of debugging + bug fixing, and we still saw unexplained behavior and crashes later on.

This is such a big surprise as we have put a lot of effort into testing over the last years. This starts from the osmo-gsm-tester software and continuously running test setup, and continues with the osmo-ttcn3-hacks integration tests that mainly I wrote during the last few months. Both us and some of our users have also (successfully!) performed interoperability testing with other vendors' implementations such as MSCs. And last, but not least, the individual Osmocom developers had been using the new post-NITB stack on their personal machines.

So what does this mean?

  • I'm sorry about the sub-standard state of the software and the resulting problems we've experienced in the 34C3 network. The extent of problems surprised me (and I presume everyone else involved)
  • I'm grateful that we've had the opportunity to discover all those bugs, thanks to the GSM team at 34C3, as well as Deutsche Telekom for donating 3 ARFCNs from their spectrum, as well as the German regulatory authority Bundesnetzagentur for providing the experimental license in the 850 MHz spectrum.
  • We need to have even more focus on automatic testing than we had so far. None of the components should be without exhaustive test coverage on at least the most common transactions, including all their failure modes (such as timeouts, rejects, ...)

My preferred method of integration testing has been by using TTCN-3 and Eclipse TITAN to emulate all the interfaces surrounding a single of the Osmocom programs (like OsmoBSC) and then test both valid and invalid transactions. For the BSC, this means emulating MS+BTS on Abis; emulating MSC on A; emulating the MGW, as well as the CTRL and VTY interfaces.

I currently see the following areas in biggest need of integration testing:

  • OsmoHLR (which needs a GSUP implementation in TTCN-3, which I've created on the spot at 34C3) where we e.g. discovered that updates to the subscriber via VTY/CTRL would surprisingly not result in an InsertSubscriberData to VLR+SGSN
  • OsmoMSC, particularly when used with external MNCC handlers, which was so far blocked by the lack of a MNCC implementation in TTCN-3, which I've been working on both on-site and after returning back home.
  • user plane testing for OsmoMGW and other components. We currently only test the control plane (MGCP), but not the actual user plane e.g. on the RTP side between the elements
  • UMTS related testing on OsmoHNBGW, OsmoMSC and OsmoSGSN. We currently have no automatic testing at all in these areas.

Even before 34C3 and the above-mentioned experiences, I concluded that for 2018 we will pursue a test-driven development approach for all new features added by the sysmocom team to the Osmocom code base. The experience with the many issues at 34C3 has just confirmed that approach. In parallel, we will have to improve test coverage on the existing code base, as outlined above. The biggest challenge will of course be to convince our paying customers of this approach, but I see very little alternative if we want to ensure production quality of our cellular stack.

So here we come: 2018, The year of testing.

December 09, 2017

Osmocom.org News: OsmoDevCon + OsmoCon - OsmoDevCon 2018 Announced

The 2018 incarnation of OsmoDevCon, the annual meeting for Osmocom developers has been scheduled for April 20-23, 2018 and will be held at the usual venue at IN-Berlin in Berlin, Germany.

For more information and Registration as well as submitting your talk proposals, see OsmoDevCon2018 and this mailing list thread

Contrary to OsmoDevCon, the date and venue for the user-oriented OsmoCon is still to be determined. It will be expanded from one to two days. Stay tuned for more news soon.

November 26, 2017

Holger "zecke" Freyther: QtVirtualKeyboard on Wayland

For the last couple of years my focus was on the Osmocom project to bring Free Software to the world of telecommunication. With a group of enthusiasts we have implemented the components necessary to run a complete network using Free Software. The Rhizomatica project is using the software to connecting people that were left behind. Our tools enabled high impact security research leading, leading to improvements to privacy and security for all of us….

But during the last months I had the opportunity to return to C++ and Qt work and it feels like coming home to the world of ARM powered hardware. When I left, the transition from consumer electronics (e.g. settop boxes) to automative (e.g. IVI) began and it seems it successfully completed! On Friday I explored a regression in OpenSSL and today I had the pleasure to understand input method handling of wayland a little bit better.

I wanted to see if I can use wayland and run QtVirtualKeyboard only in the Compositor. I couldn’t find answers in the documentation and started to read the code. Once I understood how it should work, I found a simple example in QtWayland. Isn’t Qt wonderful?

As I have gone down the rabbit hole let me try to share some of my reading. At the core is the unstable (API subject to change) text-input protocol. Upstream of wayland-protocols is still at v1 of the protocol but QtWayland includes a copy of v2 and has rather complete support for the compositor and client (thanks KDAB!).

Compositor/Server

To make it work the compositor needs to signal support for the “zwp_text_input_manager_v2” interface. In QML/Quick this needs to be done by adding the following inside the WaylandCompositor component:

...
TextInputManager {
}
...

This will instantiate a QWaylandTextInputManager and call ::initialize on it. Then auto-generated code will use wl_global_create to register the interface in the global registry. Clients will request a text_input and the manager will create one or use a QWaylandTextInput and then send events down to the client. E.g. if there is a soft keyboard set a breakpoint in QWaylandTextInput::sendKeyEvent in the compositor and see from where it is coming and how it will send the event to a client.

Client

Once the client starts the QWaylandIntegration will create a QWaylandDisplay and then query/initialize the registry. The answer will call QWaylandDisplay::registry_global for every registered interface. For the the zwp_text_input_manager_v2 interface the integration will initialize a QtWayland::zwp_text_input_manager_v2 and for each input device a QtWaylandClient::QWaylandTextInput (different namespace) will be created.

Communication

Now Compositor and Client will communicate through the wayland connection (no dbus). With Qt APIs the client will notice which element has been focused and will request the input method to be shown (QtWaylandClient::QWaylandInputContext::showInputPanel) and the Compositor  will forward this to the QInputMethod (see QWaylandTextInputPrivate::zwp_text_input_v2_show_input_panel)

Putting it together

In the end this is rather simple.. create a TextInputManager, use the InputPanel of the QtVirtualKeyboard and start your compositor with QT_IM_MODULE=qtvirtualkeyboard. Done!

 

November 12, 2017

Osmocom.org News: Cellular Network Infrastructure - Outreachy project selects Osmocom Debian Packaging

The Outreachy project has selected work on Debian packaging for Osmocom for the Dec 2017 to Mar 2018 Outreachy Interns

You can read the related announcement at the outreachy announce mailing list

Kira "kobr" Obrezkova will be working on this, with Debian developer Thorsten Alteholz as mentor.

Congratulations, Kira! Thanks to Thorsten Alteholz for mentoring as well as to Outreachy and its sponsors!

In Osmocom, we have made tremendous progress during 2016 and 2017 in re-structuring our code base, with a proper 3GPP AoIP interface between BSC and MSC, the split-up of OsmoNITB, the externalization of the HLR and full 3G integration. This has had lots of fall-out in terms of packaging, and it's important to have the new post-NITB architecture packaged properly in upstream Debian.

Outreachy provides three-month internships for people from groups traditionally underrepresented in tech. Interns are paid a stipend of $5,500 and have a $500 travel stipend available to them. Interns work remotely with mentors from Free and Open Source Software (FOSS) communities on projects ranging from programming, user experience, documentation, illustration and graphical design, to data science.

November 09, 2017

Osmocom.org News: OsmoBSC - OsmoBSC now *requires* an osmo-mgw to run alongside it

Heads up all OsmoBSC users: if you are deploying an osmo-bsc from osmo-bsc.git using the latest master branch (or nightly builds), you may notice voice streams not working anymore.

The reason is that OsmoBSC now supports intra-BSC handover (handover between separate BTS connected to the same BSC). To be able to redirect RTP streams between separate BTS, OsmoBSC now always requires an OsmoMGW instance to run alongside it.

Documentation on the Wiki and in the Manuals still needs to be updated, please bear with us until we get a chance to adjust those.

Error executing the graphviz_link macro (Missing template wiki_graphviz/macro with {:locale=>[:en], :formats=>[:atom], :variants=>[], :handlers=>[:erb, :builder, :raw, :ruby, :rsb]}. Searched in: * "/usr/local/www/redmine-3.2.9/plugins/wiki_mscgen_plugin/app/views" * "/usr/local/www/redmine-3.2.9/plugins/wiki_graphviz_plugin/app/views" * "/usr/local/www/redmine-3.2.9/plugins/redmine_wiki_extensions/app/views" * "/usr/local/www/redmine-3.2.9/plugins/redmine_openid_provider/app/views" * "/usr/local/www/redmine-3.2.9/plugins/redmine_checklists/app/views" * "/usr/local/www/redmine-3.2.9/plugins/event_notifications/app/views" * "/usr/local/www/redmine-3.2.9/app/views" )

An OsmoMGW config example is

mgcp
 bind ip 127.0.0.1
 bind port 2427
 rtp net-range 4002 16000
 number endpoints 31
 rtp-accept-all 1

If OsmoMGW is running on the same machine as OsmoBSC with MGCP at 127.0.0.1, OsmoBSC needs no further configuration and will find the OsmoMGW by default at 127.0.0.1 port 2427. More detailed OsmoBSC side config can be issued like:

msc
 mgw remote-ip 127.0.0.1
 mgw remote-port 2427
 mgw endpoint-range 1 31

You can find OsmoMGW in the nightly (and "latest") builds as well as opkg feeds, it is installed by the osmo-mgw package and developed in the osmo-mgw.git repository.

The OsmoBSC change from which on we require an OsmoMGW is here

Previously, the higher level MGW would directly talk RTP to the BTS, which is now no longer the case. The BSC will always advertise its MGW's RTP ports towards the MSC. This means that the BTS can now be in a network segment that is not reachable by the MSC directly.

November 06, 2017

Harald "LaForge" Welte: SFLC sues SFC over trademark infringement

As the Software Freedom Conservancy (SFC) has publicly disclosed on their website, it appears that Software Freedom Law Center (SFLC) has filed for a trademark infringement lawsuit against SFC.

SFLC has launched SFC in 2006, and SFLC has helped and endorsed SFC in the past.

This lawsuit is hard to believe. What has this community come to, if its various members - who used all to be respected equally - start filing law suits against each other?

It's of course not known what kind of negotiations might have happened out-of-court before an actual lawsuit has been filed. Nevertheless, one would have hoped that people are able to talk to each other, and that the mutual respect for working at different aspects and with possibly slightly different strategies would have resulted in a less confrontational approach to resolving any dispute.

To me, this story just looks like there can only be losers on all sides, by far not just limited to the two entities in question.

On lwn.net some people, including high-ranking members of the FOSS community have started to spread conspiracy theories as to whether there's any secret scheming behind the scenes, particularly from the Linux Foundation towards SFLC to cause trouble towards the SFC and their possibly-not-overly-enjoyed-by-everyone enforcement activities.

I think this is complete rubbish. Neither have I ever had the impression that the LF is completely opposed to license enforcement to begin with, nor do I have remotely enough phantasy to see them engage in such malicious scheming.

What motivates SFLC and/or Eben to attack their former offspring is however unexplainable to the bystander. One hopes there is no connection to his departure from FSF about one year ago, where he served as general counsel for more than two decades.

Harald "LaForge" Welte: On the Linux Kernel Enforcement Statement

I'm late with covering this here, but work overload is having its toll on my ability to blog.

On October 16th, key Linux Kernel developers have released and anounced the Linux Kernel Community Enforcement Statemnt.

In its actual text, those key kernel developers cover

  • compliance with the reciprocal sharing obligations of GPLv2 is critical and mandatory
  • acknowledgement to the right to enforce
  • expression of interest to ensure that enforcement actions are conducted in a manner beneficial to the larger community
  • a method to provide reinstatement of rights after ceasing a license violation (see below)
  • that legal action is a last resort
  • that after resolving any non-compliance, the formerly incompliant user is welcome to the community

I wholeheartedly agree with those. This should be no surprise as I've been one of the initiators and signatories of the earlier statement of the netfilter project on GPL enforcement.

On the reinstatement of rights

The enforcement statement then specifically expresses the view of the signatories on the specific aspect of the license termination. Particularly in the US, among legal scholars there is a strong opinion that if the rights under the GPLv2 are terminated due to non-compliance, the infringing entity needs an explicit reinstatement of rights from the copyright holder. The enforcement statement now basically states that the signatories believe the rights should automatically be re-instated if the license violation ceases within 30 days of being notified of the license violation

To people like me living in the European (and particularly German) legal framework, this has very little to no implications. It has been the major legal position that any user, even an infringing user can automatically obtain a new license as soon as he no longer violates. He just (really or imaginary) obtains a new copy of the source code, at which time he again gets a new license from the copyright holders, as long as he fulfills the license conditions.

So my personal opinion as a non-legal person active in GPL compliance on the reinstatement statement is that it changes little to nothing regarding the jurisdiction that I operate in. It merely expresses that other developers express their intent and interest to a similar approach in other jurisdictions.

October 28, 2017

Osmocom.org News: Cellular Network Infrastructure - Osmocom "latest" binary packages for Debian + Ubuntu

Starting today, Osmocom offers an osmocom:latest package feed with Ubuntu + Debian packages of the latest tagged releases of all Osmocom cellular infrastructure software.

Since early 2016, Osmocom has already been offering Nightly_Builds of the master-of-the-day of each individual projects git repository to enable users to utilize Osmocom software without having to build from source. However, by their very nature, nightly builds are volatile as they track each indiviudal development step. This is interesting for users who are testing latest developments or who need to track fixes introduced only very recently.

The new Latest_Builds only change whenever a new release tag is set in the respective source code repository, i.e. every few weeks to months for a given project. While this is not a long-terms supported release, osmocom:latest is a much more suitable choice for deployments.

October 26, 2017

Osmocom.org News: OsmoNITB - GPRS code (SGSN, GbProxy, GTPHUB) moved from openbsc.git

As part of the NITB-Split and repository reorganisation, we have moved the GPRS code that used to live in openbsc.git to a separate repository.

Technically, the OsmoSGSN, GbProxy and GTPHUB never shared any code with osmo-nitb or the other circuit-switched code in openbsc.git. It was probably a bad idea to start writing the code in the same repository at all.

Some weeks ago, we started a separate osmo-sgsn.git repository for this code, and migrated the build jobs in jenkins as well as for the Nightly_Builds over.

Effective today, the GPRS components have been removed from openbsc.git. Please use osmo-sgsn.git from now on. Sorry for any inconvenience caused.

October 19, 2017

Harald "LaForge" Welte: Obtaining the local IP address of an unbound UDP socket

Sometimes one is finding an interesting problem and is surprised that there is not a multitude of blog post, stackoverflow answers or the like about it.

A (I think) not so uncommon problem when working with datagram sockets is that you may want to know the local IP address that the OS/kernel chooses when sending a packet to a given destination.

In an unbound UDP socket, you basically send and receive packets with any number of peers from a single socket. When sending a packet to destination Y, you simply pass the destination address/port into the sendto() socket function, and the OS/kernel will figure out which of its local IP addresses will be used for reaching this particular destination.

If you're a dumb host with a single default router, then the answer to that question is simple. But in any reasonably non-trivial use case, your host will have a variety of physical and/or virtual network devices with any number of addresses on them.

Why would you want to know that address? Because maybe you need to encode that address as part of a packet payload. In the current use case that we have, it is the OsmoMGW, implementing the IETF MGCP Media Gateway Control Protocol.

So what can you do? You can actually create a new "trial" socket, not bind it to any specific local address/port, but connect() it to the destination of your IP packets. Then you do a getsockname(), which will give you the local address/port the kernel has selected for this socket. And that's exactly the answer to your question. You can now close the "trial" socket and have learned which local IP address the kernel would use if you were to send a packet to that destination.

At least on Linux, this works. While getsockname() is standard BSD sockets API, I'm not sure how portable it is to use it on a socket that has not been explicitly bound by a prior call to bind().

October 15, 2017

Holger "zecke" Freyther: Static binaries (for Go with Docker)

These days Go is quite popular for server based systems (read “cloud”) and one of the nice attributes is that compiling an application results in a single binary with no external dependencies (there is no “runtime” it has to link to). This makes deploying (read “copy to machine”) super easy and is a big contrast to something like Ruby on Rails and its thousands of dependencies. IIRC this feature was attractive to the developers of Qt’s coin (continuous integration agent) as well.

Amusingly in contrast to Rust, Swift or other modern languages the compiler/assembler/linker isn’t powered by LLVM but is based on the Plan9 C compiler which was converted to Go. By setting the GOOS and GOARCH environment variables one can easily cross-compile the binary. E.g. on MacOs build a binary that runs on Linux/ARM64.

When using other system libraries (through cgo) this single binary needs to link to other libraries but this complicates the deployment. The right version and ABI of the library need to be present, if not the application might not start or behaves weirdly. I was in this situation for my tcapflow monitoring utility. I would like to be able to deploy it on any version of RHEL, Ubuntu, Debian without anyone having to install the right libraries.

Here is where musl, Alpine and Docker came to rescue me. Let me briefly elaborate. The dominant C library on GNU/Linux is GNU Libc (glibc) doesn’t support static linking for some good (security, PIE) and some IMHO lazy reasons (PIE could still work, iconv/nss). On the other hand the musl library does support static linking and the C library is quite compatible to glibc and Alpine Linux is a Linux distribution that is using musl instead of glibc. By making use of Alpine I avoid having to build musl and then compiling libpcap and other libraries myself. The final item is Docker. It solves fetching a runnable set of binaries/libraries and setting-up/running a chroot for me. The command line below should result in the alpine container being fetched and an interactive shell prompt coming up. During development I use it to quickly fetch/try the latest version of postgres, mysql, etc.

docker run -it alpine:3.6 /bin/sh

I ended up creating a simple build script that will use the Alpine package manager to install the needed dependencies and then make a static build. The magic for the static build is to pass ldflags to go build which looks like:

go build --ldflags '-linkmode external -extldflags "-static"'

Instead of using a Dockerfile to build a container/image that I will never use (but would still consume disk space) I trigger my compilation through two commands. One to build for i386 and the other for AMD64.

docker run --rm=true -itv $PWD:/mnt alpine:3.6 /mnt/build_static.sh
docker run --rm=true -itv $PWD:/mnt i386/alpine:3.6 /mnt/build_static.sh

In the end I will have two binaries in the out/ directory of my sourcecode. I am using the Binutils objdump to look at the ELF headers of the binary to check which libraries it wants to link to. Shared library dependencies are indicated with NEEDED but in this case there is no such line which means the libpcap dependency was statically linked. For me musl+alpine+docker is the easiest way to build static binaries.

$ objdump -x out/tcapflow-client
out/tcapflow-client:     file format elf32-i386
out/tcapflow-client
architecture: i386, flags 0x00000150:
HAS_SYMS, DYNAMIC, D_PAGED
start address 0x000c55b9

Program Header:
LOAD off 0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**12
filesz 0x004ecf5c memsz 0x004ecf5c flags r-x
LOAD off 0x004edc8c vaddr 0x004eec8c paddr 0x004eec8c align 2**12
filesz 0x0032ea17 memsz 0x0075df34 flags rw-
DYNAMIC off 0x007e2f1c vaddr 0x007e3f1c paddr 0x007e3f1c align 2**2
filesz 0x000000a8 memsz 0x000000a8 flags rw-
NOTE off 0x00000120 vaddr 0x00000120 paddr 0x00000120 align 2**5
filesz 0x00000038 memsz 0x00000038 flags r--
TLS off 0x004edc8c vaddr 0x004eec8c paddr 0x004eec8c align 2**2
filesz 0x00000000 memsz 0x00000004 flags r--
STACK off 0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**4
filesz 0x00000000 memsz 0x00000000 flags rw-
RELRO off 0x004edc8c vaddr 0x004eec8c paddr 0x004eec8c align 2**0
filesz 0x002f5374 memsz 0x002f5374 flags r--

Dynamic Section:
SYMBOLIC 0x00000000
INIT 0x000c54ac
FINI 0x0046eed5
GNU_HASH 0x00000158
STRTAB 0x000001d8
SYMTAB 0x00000188
STRSZ 0x00000021
SYMENT 0x00000010
DEBUG 0x00000000
PLTGOT 0x007e3fc4
REL 0x000001fc
RELSZ 0x000c52b0
RELENT 0x00000008
BIND_NOW 0x00000000
FLAGS_1 0x08000001
RELCOUNT 0x00018a56

October 09, 2017

Harald "LaForge" Welte: Invited keynote + TTCN-3 talk at netdevconf 2.2 in Seoul

It was a big surprise that I've recently been invited to give a keynote on netfilter history at netdevconf 2.2.

First of all, I wouldn't have expected netfilter to be that relevant next to all the other [core] networking topics at netdevconf. Secondly, I've not been doing any work on netfilter for about a decade now, so my memory is a bit rusty by now ;)

Speaking of Rusty: Timing wise there is apparently a nice coincidence that I'll be able to meet up with him in Berlin later this month, i.e. hopefully we can spend some time reminiscing about old times and see what kind of useful input he has for the keynote.

I'm also asking my former colleagues and successors in the netfilter project to share with me any note-worthy events or anecdotes, particularly also covering the time after my retirement from the core team. So if you have something that you believe shouldn't miss in a keynote on netfilter project history: Please reach out to me by e-mail ASAP and let me know about it.

To try to fend off the elder[ly] statesmen image that goes along with being invited to give keynotes about the history of projects you were working on a long time ago, I also submitted an actual technical talk: TTCN-3 and Eclipse Titan for testing protocol stacks, in which I'll cover my recent journey into TTCN-3 and TITAN land, and how I think those tools can help us in the Linux [kernel] networking community to productively produce tests for the various protocols.

As usual for netdevconf, there are plenty of other exciting talks in the schedule

I'm very much looking forward to both visiting Seoul again, as well as meeting lots of the excellent people involved in the Linux networking subsystems. See ya!

October 08, 2017

Harald "LaForge" Welte: Ten years Openmoko Neo1973 release anniversary dinner

As I noted earlier this year, 2017 marks the tenth anniversary of shipping the first Openmoko phone, the Neo1973.

On this occasion, a number of the key people managed to gather for an anniversary dinner in Taipei. Thanks for everyone who could make it, it was very good to see them together again. Sadly, by far not everyone could attend. You have been missed!

The award for the most crazy attendee of the meeting goes out to my friend Milosch, who has actually flown from his home in the UK to Taiwan, only to meet up with old friends and attend the anniversary dinner.

You can some pictures in Milosch's related tweet.

October 04, 2017

Harald "LaForge" Welte: On Vacation

In case you're wondering about the lack of activity not only on this blog but also in git repositories, mailing lists and the like: I've been on vacation since September 13. It's my usual "one month in Taiwan" routine, during which I spend some time in Taipei, but also take several long motorbike tours around mostly rural Taiwan.

You can find the occasional snapshot in my twitter feed, such as the, pictures, here and there.

October 01, 2017

Kevin Redon: Which LTE band to operate on

Some might remember the beginning of mobile phones, where you could not continue using your phone when switching from one operator to another. I’m not talking about SIM locks here, which is just a software restriction, but about a physical restriction: your phone only supported one GSM band, and the new operator had only licenses for the other band. Soon phones were “dual-band”, supporting both GSM bands used in your country. But then you had the same issue again when crossing the ocean because the other continent uses yet two other GSM bands, until tri-band or quad-band phones were the norm.

September 25, 2017

Holger "zecke" Freyther: Brain dump – what fascinates me

A small brain dump of topics that currently fascinate me. These are mostly pointers and maybe it is interesting to follow it.

Books/Reading:

My kobo ebook reader has the Site Reliability Engineering book and I am now mostly done. It is kind of a revelation and explains my interest to write code but also to operate infrastructure (like struggling with ruby, rmagick, nginx…). I am interested in backends since… well ever. The first time I noticed  it when we talked about Kolab at LinuxTag and I was more interested in the backend than the KDE client. At sysmocom we built an IoT product and the backend was quite some fun, especially the scale of one instance and many devices/users, capacity planning and disk commissioning, lossless upgrades.

It can be seen in my non FOSS SS7 map work on traffic masquerading and transparent rewriting. It is also clear to see which part of engineering is needed for scale (instead of just installing and restarting servers).

Lang VM design

One technology that made Java fast (Hotspot) and has seen its way into JavaScript is dynamic optimization. Most Just in Time Compilers start with generating native code per method, either directly or after the first couple of calls when the methods size is significant enough. The VM records which call paths are hot, which types are used and then can generate optimized code (e.g. specialized for integers, remove type checks). A technique pioneered at Sun for the “Self” language (and then implemented for Strongtalk and then brought to Java) was “adaptive optimization and deoptimization” and was the Phd topic of Urs Hoelzle (Google’s VP of Engineering). One of the key aspects is inlining across method boundaries as this removes method look-up, call stack handling and opens the way for code optimization across method boundaries (at the cost of RAM usage).

In OpenJDK, V8 and JavaScriptCore this adaptive optimization is typically implemented in C++ and requires quite some code. The code is complicated as it needs to optimize but also need to return to a basic function (deoptimize, e.g. if a method changed or the types passed don’t match anymore), e.g. in the middle of a for loop with tons of inlined code (think of Array.map being inlined but then need to be de-inlined). A nice and long blog post of JSC can be found here describing the On Stack Replacement (OSR).

Long introduction and now to the new thing. In the OpensmalltalkVM a new approach called Sista has been picked and I find it is genius. Like with many problems the point of view and approach really matters. Instead of writing a lot of code in the VM the optimizer runs next to the application code. The key parts seem to be:

  • Using branches taken/not-taken as indicator how hot a path is. The overhead of counting these seem to be better than counting method calls/instructions/loops.
  • Using the Inline Caches for type information on call sites (is that mono-, poly- or megamorphic?)
  • Optimize from one set of Bytecode to another set of Bytecode.

The revelation is the last part. By just optimizing from bytecode to bytecode the VM remains in charge of creating and managing machine code. The next part is that tooling in the higher language is better or at least the roundtrip is more quick (edit code and just compile the new method instead of running make, c++, ld). The productivity thanks to the abstraction and tooling is likely higher.

As last part the OSR is easier as well. In Smalltalk thisContext (the current stack frame, activation record) is an object as well. At the right point (when the JIT has either written back variables from register to the stack or at least knows where the value is) one can just manipulate thisContext, create and link news ones and then resume execution without all the magic in other VMs.

Go, Go and escape analysis

Ken Thompson and Robert Pike are well known persons and their Go programming language is a very interesting system programming language. Like with all new languages I try to get real world experience with the language, the tooling and which kind of problems can be solved with it. I have debugged and patched some bigger projects and written two small applications with it.

There is plenty I like. The escape analysis of the compiler is fun (especially now that I know it was translated from the Plan9 C compiler from C to Go), the concurrency model is good (though allowing shared state), the module system makes sense (but makes forking harder than necessary), being able to cross compile to any target from any system.

Knowing a bit of Erlang (and continuing to read the Phd Thesis of Joe Armstrong) and being a heavy Smalltalk user there are plenty of things missing. It starts with vague runtime error messages (e.g. panicslice not having parameters) and goes to runtime and post-runtime inspection. In Smalltalk thanks to the abstraction a lot of hard things are easy and I would have wished for some of them to be in Go. Serialize all unrecovered panics? Debugging someone else’s code seems like pre 1980…

So for many developers Go is a big improvement but for some people with a wider view it might look like a lost opportunity. But that can only be felt by developers that have experienced higher abstraction and productivity.

 

Unsupervised machine learning

but that is for another dump…

September 06, 2017

Osmocom.org News: OsmoGGSN (former OpenGGSN) - OsmoGGSN succeeds OpenGGSN

12 years after OpenGGSN was seemingly abandoned by its original creators, and 7 years after Osmocom adopted it, it is time for a significant change:

OpenGGSN is becoming a first-class Osmocom citizen called OsmoGGSN.

We had already taken some baby-steps in the past by introduction of a CTRL interface as well as the use of libosmocore logging. However, my recent patches introducing a VTY interface and changing the configuration file format from the 'gengetopt' style to libosmovty based change the look+feel of the program significantly that it is a good point to rename.

After all, if command-line arguments and config file syntax are changing, documentation will also need to change and it becomes confusing to users to understand that depending on the version the documentation is correct or incorrect.

So from today on, The introduction of the VTY interface comes with many new possibilities, such as
  • multiple GGSN instances bound to different GTP IP addresses
  • multiple APNs within each GGSN, each with different Address Pools and
    tun-devices
  • sophisticated logging configuration (syslog, file, stdout, telnet)
What's still missing:
  • re-integrate kernel GTP-U support
  • create OsmoGGSN VTY reference manual
  • perl/python script to convert old config file to new config file format (any volunteers?)
Roadmap:
  • IPv6 transport plane support (outer IP layer surrounding GTP/UDP)
  • improved logging (ensure context is always included)
  • libgtp: migration of kernel GTP-U support into libgtp (not just ggsn)
  • libgtp: make PDP context hash table part of the 'gsn' structure
  • once all expected ABI/API changes are done, rename libgtp to libosmo-gtp

In terms of maintenance, I don't want to continue to maintain OpenGGSN for much longer. We'll keep it around for some time and merge important security and/or bug fixes, but I won't accept new feature patches into OpenGGSN.

September 02, 2017

Harald "LaForge" Welte: Purism Librem 5 campaign

There's a new project currently undergoing crowd funding that might be of interest to the former Openmoko community: The Purism Librem 5 campaign.

Similar to Openmoko a decade ago, they are aiming to build a FOSS based smartphone built on GNU/Linux without any proprietary drivers/blobs on the application processor, from bootloader to userspace.

Furthermore (just like Openmoko) the baseband processor is fully isolated, with no shared memory and with the Linux-running application processor being in full control.

They go beyond what we wanted to do at Openmoko in offering hardware kill switches for camera/phone/baseband/bluetooth. During Openmoko days we assumed it is sufficient to simply control all those bits from the trusted Linux domain, but of course once that might be compromised, a physical kill switch provides a completely different level of security.

I wish them all the best, and hope they can leave a better track record than Openmoko. Sure, we sold some thousands of phones, but the company quickly died, and the state of software was far from end-user-ready. I think the primary obstacles/complexities are verification of the hardware design as well as the software stack all the way up to the UI.

The budget of ~ 1.5 million seems extremely tight from my point of view, but then I have no information about how much Puri.sm is able to invest from other sources outside of the campaign.

If you're a FOSS developer with a strong interest in a Free/Open privacy-first smartphone, please note that they have several job openings, from Kernel Developer to OS Developer to UI Developer. I'd love to see some talents at work in that area.

It's a bit of a pity that almost all of the actual technical details are unspecified at this point (except RAM/flash/main-cpu). No details on the cellular modem/chipset used, no details on the camera, neither on the bluetooth chipset, wifi chipset, etc. This might be an indication of the early stage of their plannings. I would have expected that one has ironed out those questions before looking for funding - but then, it's their campaign and they can run it as they see it fit!

I for my part have just put in a pledge for one phone. Let's see what will come of it. In case you feel motivated by this post to join in: Please keep in mind that any crowdfunding campaign bears significant financial risks. So please make sure you made up your mind and don't blame my blog post for luring you into spending money :)

September 01, 2017

Harald "LaForge" Welte: First actual XMOS / XCORE project

For many years I've been fascinated by the XMOS XCore architecture. It offers a surprisingly refreshing alternative virtually any other classic microcontroller architectures out there. However, despite reading a lot about it years ago, being fascinated by it, and even giving a short informal presentation about it once, I've so far never used it. Too much "real" work imposes a high barrier to spending time learning about new architectures, languages, toolchains and the like.

Introduction into XCore

Rather than having lots of fixed-purpose built-in "hard core" peripherals for interfaces such as SPI, I2C, I2S, etc. the XCore controllers have a combination of

  • I/O ports for 1/4/8/16/32 bit wide signals, with SERDES, FIFO, hardware strobe generation, etc
  • Clock blocks for using/dividing internal or external clocks
  • hardware multi-threading that presents 8 logical threads on each core
  • xCONNECT links that can be used to connect multiple processors over 2 or 5 wires per direction
  • channels as a means of communication (similar to sockets) between threads, whether on the same xCORE or a remote core via xCONNECT
  • an extended C (xC) programming language to make use of parallelism, channels and the I/O ports

In spirit, it is like a 21st century implementation of some of the concepts established first with Transputers.

My main interest in xMOS has been the flexibility that you get in implementing not-so-standard electronics interfaces. For regular I2C, UART, SPI, etc. there is of course no such need. But every so often one encounters some interface that's very rately found (like the output of an E1/T1 Line Interface Unit).

Also, quite often I run into use cases where it's simply impossible to find a microcontroller with a sufficient number of the related peripherals built-in. Try finding a microcontroller with 8 UARTs, for example. Or one with four different PCM/I2S interfaces, which all can run in different clock domains.

The existing options of solving such problems basically boil down to either implementing it in hard-wired logic (unrealistic, complex, expensive) or going to programmable logic with CPLD or FPGAs. While the latter is certainly also quite interesting, the learning curve is steep, the tools anything but easy to use and the synthesising time (and thus development cycles) long. Furthermore, your board design will be more complex as you have that FPGA/CPLD and a microcontroller, need to interface the two, etc (yes, in high-end use cases there's the Zynq, but I'm thinking of several orders of magnitude less complex designs).

Of course one can also take a "pure software" approach and go for high-speed bit-banging. There are some ARM SoCs that can toggle their pins. People have reported rates like 14 MHz being possible on a Raspberry Pi. However, when running a general-purpose OS in parallel, this kind of speed is hard to do reliably over long term, and the related software implementations are going to be anything but nice to write.

So the XCore is looking like a nice alternative for a lot of those use cases. Where you want a microcontroller with more programmability in terms of its I/O capabilities, but not go as far as to go full-on with FPGA/CPLD development in Verilog or VHDL.

My current use case

My current use case is to implement a board that can accept four independent PCM inputs (all in slave mode, i.e. clock provided by external master) and present them via USB to a host PC. The final goal is to have a board that can be combined with the sysmoQMOD and which can interface the PCM audio of four cellular modems concurrently.

While XMOS is quite strong in the Audio field and you can find existing examples and app notes for I2S and S/PDIF, I couldn't find any existing code for a PCM slave of the given requirements (short frame sync, 8kHz sample rate, 16bit samples, 2.048 MHz bit clock, MSB first).

I wanted to get a feeling how well one can implement the related PCM slave. In order to test the slave, I decided to develop the matching PCM master and run the two against each other. Despite having never written any code for XMOS before, nor having used any of the toolchain, I was able to implement the PCM master and PCM slave within something like ~6 hours, including simulation and verification. Sure, one can certainly do that in much less time, but only once you're familiar with the tools, programming environment, language, etc. I think it's not bad.

The biggest problem was that the clock phase for a clocked output port cannot be configured, i.e. the XCore insists on always clocking out a new bit at the falling edge, while my use case of course required the opposite: Clocking oout new signals at the rising edge. I had to use a second clock block to generate the inverted clock in order to achieve that goal.

Beyond that 4xPCM use case, I also have other ideas like finally putting the osmo-e1-xcvr to use by combining it with an XMOS device to build a portable E1-to-USB adapter. I have no clue if and when I'll find time for that, but if somebody wants to join in: Let me know!

The good parts

Documentation excellent

I found the various pieces of documentation extremely useful and very well written.

Fast progress

I was able to make fast progress in solving the first task using the XMOS / Xcore approach.

Soft Cores developed in public, with commit log

You can find plenty of soft cores that XMOS has been developing on github at https://github.com/xcore, including the full commit history.

This type of development is a big improvement over what most vendors of smaller microcontrollers like Atmel are doing (infrequent tar-ball code-drops without commit history). And in the case of the classic uC vendors, we're talking about drivers only. In the XMOS case it's about the entire logic of the peripheral!

You can for example see that for their I2C core, the very active commit history goes back to January 2011.

xSIM simulation extremely helpful

The xTIMEcomposer IDE (based on Eclipse) contains extensive tracing support and an extensible near cycle accurate simulator (xSIM). I've implemented a PCM mater and PCM slave in xC and was able to simulate the program while looking at the waveforms of the logic signals between those two.

The bad parts

Unfortunately, my extremely enthusiastic reception of XMOS has suffered quite a bit over time. Let me explain why:

Hard to get XCore chips

While the product portfolio on on the xMOS website looks extremely comprehensive, the vast majority of the parts is not available from stock at distributors. You won't even get samples, and lead times are 12 weeks (!). If you check at digikey, they have listed a total of 302 different XMOS controllers, but only 35 of them are in stock. USB capable are 15. With other distributors like Farnell it's even worse.

I've seen this with other semiconductor vendors before, but never to such a large extent. Sure, some packages/configurations are not standard products, but having only 11% of the portfolio actually available is pretty bad.

In such situations, where it's difficult to convince distributors to stock parts, it would be a good idea for XMOS to stock parts themselves and provide samples / low quantities directly. Not everyone is able to order large trays and/or capable to wait 12 weeks, especially during the R&D phase of a board.

Extremely limited number of single-bit ports

In the smaller / lower pin-count parts, like the XU[F]-208 series in QFN/LQFP-64, the number of usable, exposed single-bit ports is ridiculously low. Out of the total 33 I/O lines available, only 7 can be used as single-bit I/O ports. All other lines can only be used for 4-, 8-, or 16-bit ports. If you're dealing primarily with serial interfaces like I2C, SPI, I2S, UART/USART and the like, those parallel ports are of no use, and you have to go for a mechanically much larger part (like XU[F]-216 in TQFP-128) in order to have a decent number of single-bit ports exposed. Those parts also come with twice the number of cores, memory, etc- which you don't need for slow-speed serial interfaces...

Change to a non-FOSS License

XMOS deserved a lot of praise for releasing all their soft IP cores as Free / Open Source Software on github at https://github.com/xcore. The License has basically been a 3-clause BSD license. This was a good move, as it meant that anyone could create derivative versions, whether proprietary or FOSS, and there would be virtually no license incompatibilities with whatever code people wanted to write.

However, to my very big disappointment, more recently XMOS seems to have changed their policy on this. New soft cores (released at https://github.com/xmos as opposed to the old https://github.com/xcore) are made available under a non-free license. This license is nothing like BSD 3-clause license or any other Free Software or Open Source license. It restricts the license to use the code together with an XMOS product, requires the user to contribute fixes back to XMOS and contains references to importand export control. This license is incopatible with probably any FOSS license in existance, making it impossible to write FOSS code on XMOS while using any of the new soft cores released by XMOS.

But even beyond that license change, not even all code is provided in source code format anymore. The new USB library (lib_usb) is provided as binary-only library, for example.

If you know anyone at XMOS management or XMOS legal with whom I could raise this topic of license change when transitioning from older sc_* software to later lib_* code, I would appreciate this a lot.

Proprietary Compiler

While a lot of the toolchain and IDE is based on open source (Eclipse, LLVM, ...), the actual xC compiler is proprietary.

Harald "LaForge" Welte: The sad state of voice support in cellular modems

Cellular modems have existed for decades and come in many shapes and kinds. They contain the cellular baseband processor, RF frontend, protocol stack software and anything else required to communicate with a cellular network. Basically a phone without display or input.

During the last decade or so, the vast majority of cellular modems come as LGA modules, i.e. a small PCB with all components on the top side (and a shielding can), which has contact pads on the bottom so you can solder it onto your mainboard. You can obtain them from vendors such as Sierra Wireless, u-blox, Quectel, ZTE, Huawei, Telit, Gemalto, and many others.

In most cases, the vendors now also solder those modules to small adapter boards to offer the same product in mPCIe form-factor. Other modems are directly manufactured in mPCIe or NGFF aka m.2 form-factor.

As long as those modems were still 2G / 2.5G / 2.75G, the main interconnection with the host (often some embedded system) was a serial UART. The Audio input/output for voice calls was made available as analog signals, ready to connect a microphone and spekaer, as that's what the cellular chipsets were designed for in the smartphones. In the Openmoko phones we also interfaced the audio of the cellular modem in analog, exactly for that reason.

From 3G onwards, the primary interface towards the host is now USB, with the modem running as a USB device. If your laptop contains a cellular modem, you will see it show up in the lsusb output.

From that point onwards, it would have made a lot of sense to simply expose the audio also via USB. Simply offer a multi-function USB device that has both whatever virutal serial ports for AT commands and network device for IP, and add a USB Audio device to it. It would simply show up as a "USB sound card" to the host, with all standard drivers working as expected. Sadly, nobody seems to have implemented this, at least not in a supported production version of their product

Instead, what some modem vendors have implemented as an ugly hack is the transport of 8kHz 16bit PCM samples over one of the UARTs. See for example the Quectel UC-20 or the Simcom SIM7100 which implement such a method.

All the others ignore any acess to the audio stream from software to a large part. One wonders why that is. From a software and systems architecture perspective it would be super easy. Instead, what most vendors do, is to expose a digital PCM interface. This is suboptimal in many ways:

  • there is no mPCIe standard on which pins PCM should be exposed
  • no standard product (like laptop, router, ...) with mPCIe slot will have anything connected to those PCM pins

Furthermore, each manufacturer / modem seems to support a different subset of dialect of the PCM interface in terms of

  • voltage (almost all of them are 1.8V, while mPCIe signals normally are 3.3V logic level)
  • master/slave (almost all of them insist on being a clock master)
  • sample format (alaw/ulaw/linear)
  • clock/bit rate (mostly 2.048 MHz, but can be as low as 128kHz)
  • frame sync (mostly short frame sync that ends before the first bit of the sample)
  • endianness (mostly MSB first)
  • clock phase (mostly change signals at rising edge; sample at falling edge)

It's a real nightmare, when it could be so simple. If they implemented USB-Audio, you could plug a cellular modem into any board with a mPCIe slot and it would simply work. As they don't, you need a specially designed mainboard that implements exactly the specific dialect/version of PCM of the given modem.

By the way, the most "amazing" vendor seems to be u-blox. Their Modems support PCM audio, but only the solder-type version. They simply didn't route those signals to the mPCIe slot, making audio impossible to use when using a connectorized modem. How inconvenient.

Summary

If you want to access the audio signals of a cellular modem from software, then you either

  • have standard hardware and pick one very specific modem model and hope this is available sufficiently long during your application, or
  • build your own hardware implementing a PCM slave interface and then pick + choose your cellular modem

On the Osmocom mpcie-breakout board and the sysmocom QMOD board we have exposed the PCM related pins on 2.54mm headers to allow for some separate board to pick up that PCM and offer it to the host system. However, such separate board hasn't been developed so far.

August 19, 2017

Harald "LaForge" Welte: Osmocom jenkins test suite execution

Automatic Testing in Osmocom

So far, in many Osmocom projects we have unit tests next to the code. Those unit tests are executing test on a per-C-function basis, and typically use the respective function directly from a small test program, executed at make check time. The actual main program (like OsmoBSC or OsmoBTS) is not executed at that time.

We also have VTY testing, which specifically tests that the VTY has proper documentation for all nodes of all commands.

Then there's a big gap, and we have osmo-gsm-tester for testing a full cellular network end-to-end. It includes physical GSM modesm, coaxial distribution network, attenuators, splitter/combiners, real BTS hardware and logic to run the full network, from OsmoBTS to the core - both for OsmoNITB and OsmoMSC+OsmoHLR based networks.

However, I think a lot of testing falls somewhere in between, where you want to run the program-under-test (e.g. OsmoBSC), but you don't want to run the MS, BTS and MSC that normally surroudns it. You want to test it by emulating the BTS on the Abis sid and the MSC on the A side, and just test Abis and A interface transactions.

For this kind of testing, I have recently started to investigate available options and tools.

OsmoSTP (M3UA/SUA)

Several months ago, during the development of OsmoSTP, I disovered that the Network Programming Lab of Münster University of Applied Sciences led by Michael Tuexen had released implementations of the ETSI test suite for the M3UA and SUA members of the SIGTRAN protocol family.

The somewhat difficult part is that they are implemented in scheme, using the guile interpreter/compiler, as well as a C-language based execution wrapper, which then is again called by another guile wrapper script.

I've reimplemented the test executor in python and added JUnitXML output to it. This means it can feed the test results directly into Jenkins.

I've also cleaned up the Dockerfiles and related image generation for the osmo-stp-master, m3ua-test and sua-test images, as well as some scripts to actually execute them on one of the Builders. You can find related Dockerfiles as well as associtaed Makfiles in http://git.osmocom.org/docker-playground

The end result after integration with Osmocom jenkins can be seen in the following examples on jenkins.osmocom.org for M3UA and for SUA

Triggering the builds is currently periodic once per night, but we could of course also trigger them automatically at some later point.

OpenGGSN (GTP)

For OpenGGSN, during the development of IPv6 PDP context support, I wrote some test infrastructure and test cases in TTCN-3. Those test cases can be found at http://git.osmocom.org/osmo-ttcn3-hacks/tree/ggsn_tests

I've also packaged the GGSN and the test cases each into separate Docker containers called osmo-ggsn-latest and ggsn-test. Related Dockerfiles and Makefiles can again be found in http://git.osmocom.org/docker-playground - together with a Eclipse TITAN Docker base image using Debian Stretch called debian-stretch-titan

Using those TTCN-3 test cases with the TITAN JUnitXML logger plugin we can again integrate the results directly into Jenkins, whose results you can see at https://jenkins.osmocom.org/jenkins/view/TTCN3/job/ttcn3-ggsn-test/14/testReport/(root)/GGSN_Tests/

Further Work

I've built some infrastructure for Gb (NS/BSSGP), VirtualUm and other testing, but yet have to build Docker images and related jenkins integration for it. Stay tuned about that. Also, lots more actual tests cases are required. I'm very much looking forward to any contributions.

August 18, 2017

Holger "zecke" Freyther: Creating a chroot for CentOS 7.3

I have recently written some RPM spec files (and to be honest it feels nicer than creating debian packages) and could test installing the resulting packages on a cloud based CentOS 6.8 VM. After that worked I wanted to test the package on a CentOS 7 system as well. To my surprise my cloud platform didn’t have CentOS 7 images. There was RHEL7 with extra computing costs and several CentOS images with extra packages (or “hardening”) that also incurred extra cost.

Being a Debian user for many many years I thought of using something like debootstrap. I initially remembered something called yumbootstrap but the packages/Google hits I found didn’t provide much. I mostly followed another guide and will write down some differences.

$ mkdir -p chroot/var/lib/rpm
$ rpm –rebuilddb –root=$PWD/chroot
$ rpm -i –root=$PWD/chroot –nodeps centos-release-7-3.1611.el7.centos.x86_64.rpm
$ wget -O /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7 http://mirror.centos.org/centos/7/os/x86_64/RPM-GPG-KEY-CentOS-Testing-7

# Create base7 repo
$ echo ”
[base7]
name=CentOS7
baseurl=http://mirror.centos.org/centos/7/os/x86_64/
gpgcheck=1
enabled=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7″ > /etc/yum.repos.d/CentOs7.repo

$ yum –disablerepo=\* –enablerepo=base7  –installroot=$PWD/chroot –noplugins install -y rpm-build yum

At that point one can chroot into the new directory. These were enough. I am running this on a CentOS6.8 system so some binaries might fail with the older kernel but I didn’t run into such an issue yet.

August 08, 2017

Harald "LaForge" Welte: IPv6 User Plane support in Osmocom

Preface

Cellular systems ever since GPRS are using a tunnel based architecture to provide IP connectivity to cellular terminals such as phones, modems, M2M/IoT devices and the like. The MS/UE establishes a PDP context between itself and the GGSN on the other end of the cellular network. The GGSN then is the first IP-level router, and the entire cellular network is abstracted away from the User-IP point of view.

This architecture didn't change with EGPRS, and not with UMTS, HSxPA and even survived conceptually in LTE/4G.

While the concept of a PDP context / tunnel exists to de-couple the transport layer from the structure and type of data inside the tunneled data, the primary user plane so far has been IPv4.

In Osmocom, we made sure that there are no impairments / assumptions about the contents of the tunnel, so OsmoPCU and OsmoSGSN do not care at all what bits and bytes are transmitted in the tunnel.

The only Osmocom component dealing with the type of tunnel and its payload structure is OpenGGSN. The GGSN must allocate the address/prefix assigned to each individual MS/UE, perform routing between the external IP network and the cellular network and hence is at the heart of this. Sadly, OpenGGSN was an abandoned project for many years until Osmocom adopted it, and it only implemented IPv4.

This is actually a big surprise to me. Many of the users of the Osmocom stack are from the IT security area. They use the Osmocom stack to test mobile phones for vulnerabilities, analyze mobile malware and the like. As any penetration tester should be interested in analyzing all of the attack surface exposed by a given device-under-test, I would have assumed that testing just on IPv4 would be insufficient and over the past 9 years, somebody should have come around and implemented the missing bits for IPv6 so they can test on IPv6, too.

In reality, it seems nobody appears to have shared line of thinking and invested a bit of time in growing the tools used. Or if they did, they didn't share the related code.

In June 2017, Gerrie Roos submitted a patch for OpenGGSN IPv6 support that raised hopes about soon being able to close that gap. However, at closer sight it turns out that the code was written against a more than 7 years old version of OpenGGSN, and it seems to primarily focus on IPv6 on the outer (transport) layer, rather than on the inner (user) layer.

OpenGGSN IPv6 PDP Context Support

So in July 2017, I started to work on IPv6 PDP support in OpenGGSN.

Initially I thought How hard can it be? It's not like IPv6 is new to me (I joined 6bone under 3ffe prefixes back in the 1990ies and worked on IPv6 support in ip6tables ages ago. And aside from allocating/matching longer addresses, what kind of complexity does one expect?

After my initial attempt of implementation, partially mislead by the patch that was contributed against that 2010-or-older version of OpenGGSN, I'm surprised how wrong I was.

In IPv4 PDP contexts, the process of establishing a PDP context is simple:

  • Request establishment of a PDP context, set the type to IETF IPv4
  • Receive an allocated IPv4 End User Address
  • Optionally use IPCP (part of PPP) to reques and receive DNS Server IP addresses

So I implemented the identical approach for IPv6. Maintain a pool of IPv6 addresses, allocate one, and use IPCP for DNS. And nothing worked.

  • IPv6 PDP contexts assign a /64 prefix, not a single address or a smaller prefix
  • The End User Address that's part of the Signalling plane of Layer 3 Session Management and GTP is not the actual address, but just serves to generate the interface identifier portion of a link-local IPv6 address
  • IPv6 stateless autoconfiguration is used with this link-local IPv6 address inside the User Plane, after the control plane signaling to establish the PDP context has completed. This means the GGSN needs to parse ICMPv6 router solicitations and generate ICMPV6 router advertisements.

To make things worse, the stateless autoconfiguration is modified in some subtle ways to make it different from the normal SLAAC used on Ethernet and other media:

  • the timers / lifetimes are different
  • only one prefix is permitted
  • only a prefix length of 64 is permitted

A few days later I implemented all of that, but it still didn't work. The problem was with DNS server adresses. In IPv4, the 3GPP protocols simply tunnel IPCP frames for this. This makes a lot of sense, as IPCP is designed for point-to-point interfaces, and this is exactly what a PDP context is.

In IPv6, the corresponding IP6CP protocol does not have the capability to provision DNS server addresses to a PPP client. WTF? The IETF seriously requires implementations to do DHCPv6 over PPP, after establishing a point-to-point connection, only to get DNS server information?!? Some people suggested an IETF draft to change this butthe draft has expired in 2011 and we're still stuck.

While 3GPP permits the use of DHCPv6 in some scenarios, support in phones/modems for it is not mandatory. Rather, the 3GPP has come up with their own mechanism on how to communicate DNS server IPv6 addresses during PDP context activation: The use of containers as part of the PCO Information Element used in L3-SM and GTP (see Section 10.5.6.3 of 3GPP TS 24.008. They by the way also specified the same mechanism for IPv4, so there's now two competing methods on how to provision IPv4 DNS server information: IPCP and the new method.

In any case, after some more hacking, OpenGGSN can now also provide DNS server information to the MS/UE. And once that was implemented, I had actual live uesr IPv6 data over a full Osmocom cellular stack!

Summary

We now have working IPv6 User IP in OpenGGSN. Together with the rest of the Osmocom stack you can operate a private GPRS, EGPRS, UMTS or HSPA network that provide end-to-end transparent, routed IPv6 connectivity to mobile devices.

All in all, it took much longer than nneeded, and the following questions remain in my mind:

  • why did the IETF not specify IP6CP capabilities to configure DNS servers?
  • why the complex two-stage address configuration with PDP EUA allocation for the link-local address first and then stateless autoconfiguration?
  • why don't we simply allocate the entire prefix via the End User Address information element on the signaling plane? For sure next to the 16byte address we could have put one byte for prefix-length?
  • why do I see duplication detection flavour neighbour solicitations from Qualcomm based phones on what is a point-to-point link with exactly two devices: The UE and the GGSN?
  • why do I see link-layer source address options inside the ICMPv6 neighbor and router solicitation from mobile phones, when that option is specifically not to be used on point-to-point links?
  • why is the smallest prefix that can be allocated a /64? That's such a waste for a point-to-point link with a single device on the other end, and in times of billions of connected IoT devices it will just encourage the use of non-public IPv6 space (i.e. SNAT/MASQUERADING) while wasting large parts of the address space

Some of those choices would have made sense if one would have made it fully compatible with normal IPv6 like e.g. on Ethernet. But implementing ICMPv6 router and neighbor solicitation without getting any benefit such as ability to have multiple prefixes, prefixes of different lengths, I just don't understand why anyone ever thought You can find the code at http://git.osmocom.org/openggsn/log/?h=laforge/ipv6 and the related ticket at https://osmocom.org/issues/2418

July 22, 2017

Osmocom.org News: multi-voltage USB UART - Annotated pin-out for Multivoltage UART

In order to facilitate the simpler use of the multi-voltage USB UART, an Annotated Pinout has been published.

Future PCB versions will have the signal names on the bottom layer silk screen (#2387), I'm sorry for not thinking of this for the first release already.

July 19, 2017

Osmocom.org News: Cellular Infrastructure - Virtual Um layer between BTS and MS

During the last couple of days, I've been working on completing, cleaning up and merging a Virtual Um interface (i.e. virtual radio layer) between OsmoBTS and OsmocomBB. After I started with the implementation and left it in an early stage in January 2016, Sebastian Stumpf has been completing it around early 2017, with now some subsequent fixes and improvements by me. The combined result allows us to run a complete GSM network with 1-N BTSs and 1-M MSs without any actual radio hardware, which is of course excellent for all kinds of testing scenarios.

The Virtual Um layer is based on sending L2 frames (blocks) encapsulated via GSMTAP UDP multicast packets. There are two separate multicast groups, one for uplink and one for downlink. The multicast nature simulates the shared medium and enables any simulated phone to receive the signal from multiple BTSs via the downlink multicast group.

Error executing the graphviz_link macro (Missing template wiki_graphviz/macro with {:locale=>[:en], :formats=>[:atom], :variants=>[], :handlers=>[:erb, :builder, :raw, :ruby, :rsb]}. Searched in: * "/usr/local/www/redmine-3.2.9/plugins/wiki_mscgen_plugin/app/views" * "/usr/local/www/redmine-3.2.9/plugins/wiki_graphviz_plugin/app/views" * "/usr/local/www/redmine-3.2.9/plugins/redmine_wiki_extensions/app/views" * "/usr/local/www/redmine-3.2.9/plugins/redmine_openid_provider/app/views" * "/usr/local/www/redmine-3.2.9/plugins/redmine_checklists/app/views" * "/usr/local/www/redmine-3.2.9/plugins/event_notifications/app/views" * "/usr/local/www/redmine-3.2.9/app/views" )

In OsmoBTS, this is implemented via the new osmo-bts-virtual BTS model.

In OsmocomBB, this is realized by adding virtphy virtual L1, which speask the same L1CTL protocol that is used between the real OsmcoomBB Layer1 and the Layer2/3 programs such as Mobile and the like.

Now many people would argue that GSM without the radio and actual handsets is no fun. I tend to agree, as I'm a hardware person at heart and I am not a big fan of simulation.

Nevertheless, this forms the basis of all kinds of possibilities for automatized (regression) testing in a way and for layers/interfaces that osmo-gsm-tester cannot cover as it uses a black-box proprietary mobile phone (modem). It is also pretty useful if you're travelling a lot and don't want to carry around a BTS and phones all the time, or get some development done in airplanes or other places where operating a radio transmitter is not really a (viable) option.

If you're curious and want to give it a shot, I've put together some setup instructions at Virtual Um.

July 18, 2017

Harald "LaForge" Welte: Virtual Um interface between OsmoBTS and OsmocomBB

During the last couple of days, I've been working on completing, cleaning up and merging a Virtual Um interface (i.e. virtual radio layer) between OsmoBTS and OsmocomBB. After I started with the implementation and left it in an early stage in January 2016, Sebastian Stumpf has been completing it around early 2017, with now some subsequent fixes and improvements by me. The combined result allows us to run a complete GSM network with 1-N BTSs and 1-M MSs without any actual radio hardware, which is of course excellent for all kinds of testing scenarios.

The Virtual Um layer is based on sending L2 frames (blocks) encapsulated via GSMTAP UDP multicast packets. There are two separate multicast groups, one for uplink and one for downlink. The multicast nature simulates the shared medium and enables any simulated phone to receive the signal from multiple BTSs via the downlink multicast group.

/images/osmocom-virtum.png

In OsmoBTS, this is implemented via the new osmo-bts-virtual BTS model.

In OsmocomBB, this is realized by adding virtphy virtual L1, which speaks the same L1CTL protocol that is used between the real OsmcoomBB Layer1 and the Layer2/3 programs such as mobile and the like.

Now many people would argue that GSM without the radio and actual handsets is no fun. I tend to agree, as I'm a hardware person at heart and I am not a big fan of simulation.

Nevertheless, this forms the basis of all kinds of possibilities for automatized (regression) testing in a way and for layers/interfaces that osmo-gsm-tester cannot cover as it uses a black-box proprietary mobile phone (modem). It is also pretty useful if you're traveling a lot and don't want to carry around a BTS and phones all the time, or get some development done in airplanes or other places where operating a radio transmitter is not really a (viable) option.

If you're curious and want to give it a shot, I've put together some setup instructions at the Virtual Um page of the Osmocom Wiki.

July 09, 2017

Harald "LaForge" Welte: Ten years after first shipping Openmoko Neo1973

Exactly 10 years ago, on July 9th, 2007 we started to sell+ship the first Openmoko Neo1973. To be more precise, the webshop actually opened a few hours early, depending on your time zone. Sean announced the availability in this mailing list post

I don't really have to add much to my ten years [of starting to work on] Openmoko anniversary blog post a year ago, but still thought it's worth while to point out the tenth anniversary.

It was exciting times, and there was a lot of pioneering spirit: Building a Linux based smartphone with a 100% FOSS software stack on the application processor, including all drivers, userland, applications - at a time before Android was known or announced. As history shows, we'd been working in parallel with Apple on the iPhone, and Google on Android. Of course there's little chance that a small taiwanese company can compete with the endless resources of the big industry giants, and the many Neo1973 delays meant we had missed the window of opportunity to be the first on the market.

It's sad that Openmoko (or similar projects) have not survived even as a special-interest project for FOSS enthusiasts. Today, virtually all options of smartphones are encumbered with way more proprietary blobs than we could ever imagine back then.

In any case, the tenth anniversary of trying to change the amount of Free Softwware in the smartphone world is worth some celebration. I'm reaching out to old friends and colleagues, and I guess we'll have somewhat of a celebration party both in Germany and in Taiwan (where I'll be for my holidays from mid-September to mid-October).

July 06, 2017

Holger "zecke" Freyther: Funding the Osmocom Cellular project

My friend and business partner has recently blogged about funding of the Osmocom Cellular Infrastructure Projects and while I want to write about the history of sysmocom s.f.m.c. GmbH I will focus on getting contributions (or as a replacement monetary support) for the project.

First of all I think the existence of Osmocom and Osmocom Cellular made a significant difference. It is used to provide connectivity to those previously ignored (Thank you everyone involved with Rhizomatica!) and we enabled mobile communication security research. This ranges from breaking ciphering, hijacking calls, easily fuzzing phones, the whole set of GSM MAP/CAP hacks which lead to real improvement of security and privacy for end users. We took the black out of the mobile black box and want to continue to do it.

My big question is how do we sustain such development (beyond personal sacrifice)? How do we get significant contributions to remove more black boxes and extend to 4G and beyond? If getting contributions is difficult the second best thing seems to be money. This allows to pay and hire new developers that want to spend their work hours on improving Free Software. So where can these contributions come from?

The research/security community

While OsmocomBB and OpenBSC opened up the door for university and corporate researchers to explore networks, offer penetration tests, the project didn’t get much in return though. Part of the problem seems that for research a sloppy modification is enough and when the researcher has published his paper, he is too ashamed to release the hack and moves on.

Universities and Students

Universities used to buy full GSM BTS but recently seem more interested in SDR platforms. While a SDR is not a BTS the promise of running a GSM and LTE network with the same universal radio peripheral is tempting. Fewer BTS sold means less funding for OpenBSC/osmo-bts but this could be easily compensated by increased contributions to osmo-bts and osmo-trx by students and university staff. For some reason this is not happening and I think there are plenty things to improve!

Vendors using OpenBSC and osmo-bts

In general I would expect that BTS vendors that integrate our software with their hardware would have an interest in the longevity of the project and either buy software support or have their staff maintain and contribute fixes. Sadly it seems that with the current state of the industry not contributing is seen as a commercial advantage…

Research grants

The first time I heard of funding of a Free Software project receiving significant funding was when the PyPy project was initiated. Today there are various funds that support Free Software initiatives (NLnet, Mozilla Grants and more) and last year my proposal to NLnet was selected and sysmocom could begin work on 3G support in Osmocom. While this is great, the amount of funding is not enough to keep a company focused on removing blackboxes from mobile communication going for too long. So more and bigger funds are needed.

I tried to get funds from Opentech but they didn’t seem to be interested in projects like replacing proprietary Qualcomm components from modules like the EC20/EC25, or building tools for 2G/3G/4G to allow to educate users on privacy impacts of using cellular technology and to understand how a phone behaves. My first research question would be to explore what really happens when 2G is disabled in a phone and a network tries to force a downgrade. But the proposal would have enabled much more. The proposals were rejected, maybe my proposal was just bad, maybe there is no interest to finance work on cellular technology (besides most data usage seems to be from mobile devices these days). The rejection doesn’t contain feedback so it is hard to tell which of the above is more true.

How can you help?

Maybe there is not enough interest and we should focus our time and energy somewhere else but if you consider our work as important as we do, maybe you can help us? We are looking

  • contributions fix a bug, add a feature, improve existing work and make sure it gets integrated
  • Help us to write project proposals for funds like the Opentech fund…
  • Buy sysmocom hardware?
  • Buy a moral license if your company can/want to do that?
  • Sponsor me (or someone else) and send bitcoin (?)?
  • Propose your idea?

June 24, 2017

Osmocom.org News: OpenBSC - Lab Update: OsmoMSC Serves 2G + 3G for the First Time

Yesterday we've reached a remarkable milestone: the new OsmoMSC has first subscribed a 3G as well as a 2G phone at the same time!

Recall the recent big developments in Osmocom:

  • creating OsmoHLR to manage subscribers asynchronously and across voice and data realms,
  • separating an OsmoMSC off OsmoNITB,
  • creating a true asynchronous state machine driven VLR in OsmoMSC,
  • adding UMTS authentication with Milenage,
  • supporting IuCS (and IuPS) to enable hNodeB driven 3G in Osmocom,
  • and last but not least adding a true A interface to OsmoMSC using our brand new SCCP/M3UA impementation.

After this work has reached a stage where we can subscribe phones, send SMS and call each other using AoverIP and 3G separately, the remaining big step was to combine all of this in the new OsmoMSC: can we talk both A over IP to our separate OsmoBSC as well as IuCS via OsmoHNBGW to a 3G hNodeB, at the same time?

Some patches are still in the queue, but since yesterday, the answer is a resounding: Yes!

Typical for a software engineer's mindset, the joy of reaching this milestone is immediately followed by an outlook of what is left open:

  • Split the current / legacy openbsc.git repository in separate new projects and lay the OsmoNITB to rest.
  • Rename our MGCP gateway (osmo-bsc_mgcp) to OsmoMGW and teach it to transcode between TRAU frames, RTP and the 3G IuUP to facilitate voice calls between all of legacy BTS models using E1, our "current" 2G BTSes talking RTP over IP as well as 3G hNodeBs that encapsulate IuUP in RTP.
  • Polish to production quality, update the docs and package for various platforms.

These are exciting times to be part of Osmocom: big changes are finally converging, to open up new horizons for FOSS driven cellular network technology.

June 15, 2017

Harald "LaForge" Welte: How the Osmocom GSM stack is funded

As the topic has been raised on twitter, I thought I might share a bit of insight into the funding of the Osmocom Cellular Infrastructure Projects.

Keep in mind: Osmocom is a much larger umbrella project, and beyond the Networks-side cellular stack is home many different community-based projects around open source mobile communications. All of those have started more or less as just for fun projects, nothing serious, just a hobby [1]

The projects implementing the network-side protocol stacks and network elements of GSM/GPRS/EGPRS/UMTS cellular networks are somewhat the exception to that, as they have evolved to some extent professionalized. We call those projects collectively the Cellular Infrastructure projects inside Osmocom. This post is about that part of Osmocom only

History

From late 2008 through 2009, People like Holger and I were working on bs11-abis and later OpenBSC only in our spare time. The name Osmocom didn't even exist back then. There was a strong technical community with contributions from Sylvain Munaut, Andreas Eversberg, Daniel Willmann, Jan Luebbe and a few others. None of this would have been possible if it wasn't for all the help we got from Dieter Spaar with the BS-11 [2]. We all had our dayjob in other places, and OpenBSC work was really just a hobby. People were working on it, because it was where no FOSS hacker has gone before. It was cool. It was a big and pleasant challenge to enter the closed telecom space as pure autodidacts.

Holger and I were doing freelance contract development work on Open Source projects for many years before. I was mostly doing Linux related contracting, while Holger has been active in all kinds of areas throughout the FOSS software stack.

In 2010, Holger and I saw some first interest by companies into OpenBSC, including Netzing AG and On-Waves ehf. So we were able to spend at least some of our paid time on OpenBSC/Osmocom related contract work, and were thus able to do less other work. We also continued to spend tons of spare time in bringing Osmocom forward. Also, the amount of contract work we did was only a fraction of the many more hours of spare time.

In 2011, Holger and I decided to start the company sysmocom in order to generate more funding for the Osmocom GSM projects by means of financing software development by product sales. So rather than doing freelance work for companies who bought their BTS hardware from other places (and spent huge amounts of cash on that), we decided that we wanted to be a full solution supplier, who can offer a complete product based on all hardware and software required to run small GSM networks.

The only problem is: We still needed an actual BTS for that. Through some reverse engineering of existing products we figured out who one of the ODM suppliers for the hardware + PHY layer was, and decided to develop the OsmoBTS software to do so. We inherited some of the early code from work done by Andreas Eversberg on the jolly/bts branch of OsmocomBB (thanks), but much was missing at the time.

What follows was Holger and me working several years for free [3], without any salary, in order to complete the OsmoBTS software, build an embedded Linux distribution around it based on OE/poky, write documentation, etc. and complete the first sysmocom product: The sysmoBTS 1002

We did that not because we want to get rich, or because we want to run a business. We did it simply because we saw an opportunity to generate funding for the Osmocom projects and make them more sustainable and successful. And because we believe there is a big, gaping, huge vacuum in terms of absence of FOSS in the cellular telecom sphere.

Funding by means of sysmocom product sales

Once we started to sell the sysmoBTS products, we were able to fund Osmocom related development from the profits made on hardware / full-system product sales. Every single unit sold made a big contribution towards funding both the maintenance as well as the ongoing development on new features.

This source of funding continues to be an important factor today.

Funding by means of R&D contracts

The probably best and most welcome method of funding Osmocom related work is by means of R&D projects in which a customer funds our work to extend the Osmocom GSM stack in one particular area where he has a particular need that the existing code cannot fulfill yet.

This kind of project is the ideal match, as it shows where the true strength of FOSS is: Each of those customers did not have to fund the development of a GSM stack from scratch. Rather, they only had to fund those bits that were missing for their particular application.

Our reference for this is and has been On-Waves, who have been funding development of their required features (and bug fixing etc.) since 2010.

We've of course had many other projects from a variety of customers over over the years. Last, but not least, we had a customer who willingly co-funded (together with funds from NLnet foundation and lots of unpaid effort by sysmocom) the 3G/3.5G support in the Osmocom stack.

The problem here is:

  • we have not been able to secure anywhere nearly as many of those R&D projects within the cellular industry, despite believing we have a very good foundation upon which we can built. I've been writing many exciting technical project proposals
  • you almost exclusively get funding only for new features. But it's very hard to get funding for the core maintenance work. The bug-fixing, code review, code refactoring, testing, etc.

So as a result, the profit margin you have on selling R&D projects is basically used to (do a bad job of) fund those bits and pieces that nobody wants to pay for.

Funding by means of customer support

There is a way to generate funding for development by providing support services. We've had some success with this, but primarily alongside the actual hardware/system sales - not so much in terms of pure software-only support.

Also, providing support services from a R&D company means:

  • either you distract your developers by handling support inquiries. This means they will have less time to work on actual code, and likely get side tracked by too many issues that make it hard to focus
  • or you have to hire separate support staff. This of course means that the size of the support business has to be sufficiently large to not only cover the cots of hiring + training support staff, but also still generate funding for the actual software R&D.

We've tried shortly with the second option, but fallen back to the first for now. There's simply not sufficient user/admin type support business to rectify dedicated staff for that.

Funding by means of cross-subsizing from other business areas

sysmocom also started to do some non-Osmocom projects in order to generate revenue that we can feed again into Osmocom projects. I'm not at liberty to discuss them in detail, but basically we've been doing pretty much anything from

  • custom embedded Linux board designs
  • M2M devices with GSM modems
  • consulting gigs
  • public tendered research projects

Profits from all those areas went again into Osmocom development.

Last, but not least, we also operate the sysmocom webshop. The profit we make on those products also is again immediately re-invested into Osmocom development.

Funding by grants

We've had some success in securing funding from NLnet Foundation for specific features. While this is useful, the size of their projects grants of up to EUR 30k is not a good fit for the scale of the tasks we have at hand inside Osmocom. You may think that's a considerable amount of money? Well, that translates to 2-3 man-months of work at a bare cost-covering rate. At a team size of 6 developers, you would theoretically have churned through that in two weeks. Also, their focus is (understandably) on Internet and IT security, and not so much cellular communications.

There are of course other options for grants, such as government research grants and the like. However, they require long-term planning, they require you to match (i.e. pay yourself) a significant portion, and basically mandate that you hire one extra person for doing all the required paperwork and reporting. So all in all, not a particularly attractive option for a very small company consisting of die hard engineers.

Funding by more BTS ports

At sysmocom, we've been doing some ports of the OsmoBTS + OsmoPCU software to other hardware, and supporting those other BTS vendors with porting, R&D and support services.

If sysmocom was a classic BTS vendor, we would not help our "competition". However, we are not. sysmocom exists to help Osmocom, and we strongly believe in open systems and architectures, without a single point of failure, a single supplier for any component or any type of vendor lock-in.

So we happily help third parties to get Osmocom running on their hardware, either with a proprietary PHY or with OsmoTRX.

However, we expect that those BTS vendors also understand their responsibility to share the development and maintenance effort of the stack. Preferably by dedicating some of their own staff to work in the Osmocom community. Alternatively, sysmocom can perform that work as paid service. But that's a double-edged sword: We don't want to be a single point of failure.

Osmocom funding outside of sysmocom

Osmocom is of course more than sysmocom. Even for the cellular infrastructure projects inside Osmocom is true: They are true, community-based, open, collaborative development projects. Anyone can contribute.

Over the years, there have been code contributions by e.g. Fairwaves. They, too, build GSM base station hardware and use that as a means to not only recover the R&D on the hardware, but also to contribute to Osmocom. At some point a few years ago, there was a lot of work from them in the area of OsmoTRX, OsmoBTS and OsmoPCU. Unfortunately, in more recent years, they have not been able to keep up the level of contributions.

There are other companies engaged in activities with and around Osmcoom. There's Rhizomatica, an NGO helping indigenous communities to run their own cellular networks. They have been funding some of our efforts, but being an NGO helping rural regions in developing countries, they of course also don't have the deep pockets. Ideally, we'd want to be the ones contributing to them, not the other way around.

State of funding

We're making some progress in securing funding from players we cannot name [4] during recent years. We're also making occasional progress in convincing BTS suppliers to chip in their share. Unfortunately there are more who don't live up to their responsibility than those who do. I might start calling them out by name one day. The wider community and the public actually deserves to know who plays by FOSS rules and who doesn't. That's not shaming, it's just stating bare facts.

Which brings us to:

  • sysmocom is in an office that's actually too small for the team, equipment and stock. But we certainly cannot afford more space.
  • we cannot pay our employees what they could earn working at similar positions in other companies. So working at sysmocom requires dedication to the cause :)
  • Holger and I have invested way more time than we have ever paid us, even more so considering the opportunity cost of what we would have earned if we'd continued our freelance Open Source hacker path
  • we're [just barely] managing to pay for 6 developers dedicated to Osmocom development on our payroll based on the various funding sources indicated above

Nevertheless, I doubt that any such a small team has ever implemented an end-to-end GSM/GPRS/EGPRS network from RAN to Core at comparative feature set. My deepest respects to everyone involved. The big task now is to make it sustainable.

Summary

So as you can see, there's quite a bit of funding around. However, it always falls short of what's needed to implement all parts properly, and even not quite sufficient to keep maintaining the status quo in a proper and tested way. That can often be frustrating (mostly to us but sometimes also to users who run into regressions and oter bugs). There's so much more potential. So many things we wanted to add or clean up for a long time, but too little people interested in joining in, helping out - financially or by writing code.

On thing that is often a challenge when dealing with traditional customers: We are not developing a product and then selling a ready-made product. In fact, in FOSS this would be more or less suicidal: We'd have to invest man-years upfront, but then once it is finished, everyone can use it without having to partake in that investment.

So instead, the FOSS model requires the customers/users to chip in early during the R&D phase, in order to then subsequently harvest the fruits of that.

I think the lack of a FOSS mindset across the cellular / telecom industry is the biggest constraining factor here. I've seen that some 20-15 years ago in the Linux world. Trust me, it takes a lot of dedication to the cause to endure this lack of comprehension so many years later.

[1]just like Linux has started out.
[2]while you will not find a lot of commits from Dieter in the code, he has been playing a key role in doing a lot of prototyping, reverse engineering and debugging!
[3]sysmocom is 100% privately held by Holger and me, we intentionally have no external investors and are proud to never had to take a bank loan. So all we could invest was our own money and, most of all, time.
[4]contrary to the FOSS world, a lot of aspects are confidential in business, and we're not at liberty to disclose the identities of all our customers

Copyright (C) 2001-2010 by the respective authors.