main.asciidoc

Interoperability between classic infrastructure and Libre-Mesh networks in Guifi.net

Introduction

The biggest community network in the world is Guifi.net [ref:cnw] it spreads across Catalonia and beyond. IP routing on a such big community network is not a trivial task, in addition to the big size, the decentralized nature of the network encourage choice, so usually different parts of Guifi.net network uses different routing protocols.
In order to make those networks parts interoperate is now mostly a manual job, various groups have developed several OpenWrt based solutions to ease networking equipment configuration, but they didn’t have achieved seamless protocol interoperability, moreover Guifi.net is mostly built with low cost embedded devices, so little performance footprint is always a requirement.
The scope of this work is to investigate mainly used routing protocols in particular BGP, BMX6 and BATMAN-adv, to study past interoperability experiments and to extend Libre-Mesh to implement seamless interoperability between classic infrastructure and Libre-Mesh networks in Guifi.net.
Bringing the new features at an usable and stable quality, testing them in real environment, protocols parameters adjusting for the optimal functioning and their deployment in production setups such as Barcelona Botanical Garden, AureaSocial offices and others clients are parts of this work too.

State of the art

BGP

BGP (acronym of Border Gateway Protocol) is a standard routing protocol designed to exchange routing and reachability information between autonomous systems (AS) on the Internet. BGP makes routing decisions based on paths, network policies, or rule-sets configured by a network administrator that is involved in making core routing decisions, because it needs information of each route reachability path to make decisions BGP is usually classified as a path vector protocol. Version 4 of BGP has been in use on the Internet since 1994, it’s mayor improvement in respect to older version that are obsolete and unused on the Internet is the support for classless inter-domain routing (CIDR) and routes aggregation.
BGP may be used for routing within an AS. In this application it is referred to as Internal BGP, or iBGP. In contrast, the Internet application of the protocol may be referred to as Exterior Border Gateway Protocol, External BGP, or eBGP.
As BGP doesn’t perform any type of neighbor discovery, peering are setup by manual configuration. For each peer router a connection through TCP port 179 must be established, after the connection is established BGP daemons synchronize with peers by sending the whole routing information they have to peers, after this initial sync only updates are transmitted to peers.

Guifi.net

Guifi.net networks uses different routing protocols such as BGP, OSPF, OLSR, BMX6, BATMAN-adv, Babel and more. Over time lot of Guifi.net nodes have been build sticking to the older but well known solution named Modelo Híbrido. This solution imply lot of manual work and a central provisioning server that know all details about the Guifi.net network, de facto limiting the resilience, the scalability and the growth of Guifi.net. To overcome Modelo Híbrido limitations lot of efforts developing new solutions have existed most of them based on mesh technology but none of them, mainly because of resource lack and consequent missing long term support, have been widely adopted causing even more fragmentation and making more difficult interoperability. Even if Libre-Mesh has been bred independently, it is already used in some Guifi.net zones, but like other mesh based solution it is at moment used only for last mile distribution.

Modelo Híbrido

As of Modelo Híbrido community network nodes can belong to two categories: supernodos and nodos, in this model nodos are not really nodes of the network as they do not do routing for other nodes of Guifi.net and just attach to a supernodos, while supernodos do route [ref:mdhdspnd]. In supernodos usually a Mikrotik routerboard is used as router while externals WiFi AP are configured as plain WDS bridge [ref:mdhd], if two supernodos are connected by a WiFi link the router of each node see the router of the other node as if it was directly connected with an Ethernet cable. On top of the routers of each node BGP routing protocol is running, on each node connected BGP peering have to be established, each supernodo has his own unique private ASN assigned by the Guifi.net Drupal based network map, in this setup each supernodo of Guifi.net act as an AS with typical eBGP information exchange mode.
While BPG seems the more appropriate routing protocol for a neutral Internet exchange point, wireless community networks have completely different needs that BGP cannot satisfy such as choosing WiFi path with more bandwidth available or automatic link establishment. Modelo Híbrido has helped Guifi.net to grow but over time new technology better suited for wireless community have emerged exposing many limitations of this model.

BIRD

BIRD is an open source implementation of multiple Internet routing protocols in form of a daemon for Unix like systems. It is distributed under the GNU General Public License. BIRD supports IPv4 or IPv6 as separate daemons [ref:birdipvx], multiple routing tables [ref:birdmultitable], and BGP, RIP and OSPF routing protocols, as well as statically defined routes. Its design differs significantly from the better known routing daemons, GNU Zebra and Quagga [ref:birdwikipedia]. Currently BIRD is included in many Linux distributions [ref:birdwikipedia] in particular into OpenWrt [ref:birdowrt].
BIRD provide very powerful route filtering framework including a simple C like programming language. There are two objects in this language: filters and functions. Filters are interpreted by BIRD core when a route is being passed between protocols and routing tables. The filter language contains control structures such as if and case, but it allows no loops. Filter gets the route, can read and write its attributes as needed and at the end, decides whether to pass the changed route through or whether to reject it. BIRD functions are meant to avoid code repetition. Functions can have parameters and they can define local variables. Recursion is not allowed. [ref:birdfilters]
BIRD is notorious for his good performances and for his lightweight design because of this it is used in several Internet exchanges as a route server [ref:birdwikipedia], where it replaced Quagga because of its scalability issues [ref:birdlinx].

BMX6

BMX6 is an open-source, mesh routing protocol, developed by Axel Neumann. It originated as an independent branch of the BATMAN routing protocol and is already used successfully in mesh networks [ref:bmxan]. Being a mesh routing protocol, BMX6 distributes node descriptions which can contain node-specific configurations. It optimizes the communication between neighboring nodes for minimal protocol overhead. From a routing protocol point of view BMX6 is classified as table-driven proactive, distance vector routing protocol so even if there is no traffic in the network, the routes to all the network nodes are known by all nodes [ref:tablepro], while nodes does not have a global view of the network topology, so their knowledge is limited to next-hop to all destinations and the distance associated to it [ref:distvect].

B.A.T.M.A.N. advanced

B.A.T.M.A.N. advanced (often referenced as batman-adv) is an implementation of the B.A.T.M.A.N. routing protocol in form of a Linux kernel module operating on layer 2. While usually wireless routing protocol implementations operates on L3 which means they exchange routing information by sending IP packets and bring their routing decision into effect by manipulating the kernel routing table. Batman-adv operates entirely on L2, not only the routing information is transported using raw Ethernet frames but also the data traffic is handled by batman-adv. It encapsulates and forwards all traffic until it reaches the destination, hence emulating a virtual network switch of all nodes participating. Therefore to the upper level all nodes appear to be link local and are unaware of the network’s topology as well as unaffected by lower level network changes. Operating as a L2 routing protocol as there is no L2 routing table to manipulate, data frames must be processed directly by batman-adv and doing it in user-space would be very inefficient, as for each packet processed it would be read() and write() to the kernel and back causing multiple context switching that are computationally costly [ref:contextswitch], which limits the sustainable bandwidth especially on low-end devices. To have good support for these devices as well and an overall good efficiency, batman-adv is implemented as a Linux kernel module. This way batman-adv avoid the context switching costs and only introduces a negligible packet processing overhead even under a high load. [ref:batmanadv]

qMp

qMp do facilitate the creation of roaming enabled mesh networks, it is based on OpenWrt and BMX6 [ref:qmp], qMp already supports configuration provisioning by guifi.net map unsolclick for mesh parameters adjusting [ref:qmpunsolclick] but it leave the interoperability with Modelo Híbrido mostly to manual static configuration done by the user [ref:qmpmdhd] or in rare cases to manually configured Quagga-BMX6 module that is unreliable, resource intensive and requires manual configuration. Moreover qMp is designed around BMX6, in a way that to add support for another routing protocol would imply a big refactoring of qMp code.

Libre-Mesh

Libre-Mesh (often referenced as LiMe) is an initiative undertaken by community networks members of several continents that collaborate towards a common goal: to develop a set of tools that facilitate the deployment of Free Networks for any community in the world.
Main tool is LiMe meta-firmware: based on OpenWrt, eases the creation of community networks, and enables existing communities to add roaming enabled network clouds to their networks.
To accomplish his goal LiMe meta-firmware have an highly modular design and implementation, the core of LiMe meta-firmware is the lime-system package.

Network Architecture

The LiMe network architecture is a communication network of radio and cabled nodes organized in a mesh-like topology. LiMe network architecture has lot in common to wireless mesh network [ref:wkdpwmn] but it provide further optimization and scalability taking advantage of the layered structure of the Internet protocol suite [ref:wkpfinternetprotocolsuite]. The creation of B.A.T.M.A.N. advanced has fostered the debate in community networks if it is better to build the mesh network with L2 routing protocols (like batman-adv) or with L3 routing protocols (like Babel, BMX6, OLSR). LiMe developers recognize the bad and the good parts of both worlds (L2/L3) and have decided to get the best of them. While LiMe let the community choice what protocol they prefer to use the default distribution include both batman-adv and BMX6 operating at same time but with different objectives.

L2 cloud

LiMe developers refers to L2 cloud (or just cloud) as a community network part in which the same broadcast domain is shared. A client connected to a cloud behave like if it is connected on the same Ethernet switch [ref:wkpdethswitch] together with all other hosts in the same cloud, all applications that take advantage of being connected to a LAN (as example: for neighbor discovery) can be used seamlessly on a cloud. As the broadcast domain is shared a client can roam without upper layer connectivity interruption across nodes of the same cloud because the assigned IP is still valid and the gateway is still “directly” reachable. On a cloud because all hosts share the same broadcast domain broadcast packets like ARP requests are flooded to the whole cloud [ref:wkpdbroadcstdoimain], this pose limits to the scalability of a cloud, moreover sharing the same broadcast domain mean that typical L2 attacks such as ARP spoofing are possible [ref:wkpdarpspoofing]. Those two drawbacks of operating at L2 limits de facto the dimension of a cloud. Despite LiMe developers suggests to avoid to build too big L2 clouds around the world exists community networks using LiMe with L2 clouds spreading on entire small cities without noticing critical problems.

Figure 1. Simplified LiMe L2 cloud schema of Barcelona Botanical garden network.

Legenda

• Black routers: LiMe router that offer WiFi connectivity.

• Blue lines: Mesh links between routers.

• Persons with smartphone: Visitors accessing “Guia Multimèdia del Jardí Botànic” geolocalization capable web application via the WiFi connectivity provided by Libre-Mesh.

L3 network

LiMe developers refers to L3 network (or just network) as a community network part that may contain multiple broadcast domain and in which packet routing between them is done at L3. From the start LiMe has been bred with IPv6 as L3 reference protocol in mind but as of today important applications and a big part of Internet still rely on IPv4 support so LiMe does support both of them implementing a complete dual stack setup. Each broadcast domain must use one or more unique subnets that doesn’t overlap with subnets used in other broadcast domains of the same network. Another key concept of LiMe is to build horizontal networks in which there is no discrimination between clients and servers. On a LiMe network every host connected can at same time offer services or be client of services offered by other hosts, because of that bandwidth offered to hosts is symmetric and NAT usage is banned.

Some of NAT drawbacks

• In presence of NAT direct connection between hosts is difficult, to open a connection special techniques are required [ref:wkpdholepunch].

• NAT break the end-to-end principle [ref:wkpfnat][ref:wkpdend2end].

• NAT based network are unfair because the natted host lose the opportunity to be a server (listening for incoming connections), and the non natted automatically acquire the privilege of listening for connections that isn’t fairly distributed to all hosts anymore.

Figure 2. Connection tentative frustrated by NAT in an IPv4 network.

Legenda

• Blue lines: Mesh links between routers.

• Purple directed lines: Upper layer connections.

• Smart-phone 192.0.2.4: It can open connections towards non natted hosts like 198.51.100.3, but cannot receive connections because they get frustrated by NAT on his router.

• Smart-phone 198.51.100.3: It can receive connections from all hosts. It can open connections to all hosts but the the one towards the other smart-phone get frustrated by NAT.

• Server 203.0.113.34: Same as smart-phone 198.51.100.3.

• Documentation addresses conform to RFC 5737 [ref:rfc5737].

Although the choice of banning NAT from LiMe networks may be imputed to courageous developers it is to be considered as a pragmatic choice too as networks without NAT has better performances [ref:natperformance], upper layer applications doesn’t have to deal with lower layer asymmetry introduced by NAT, and network debugging is way simpler.

Figure 3. Connection tentative between clients works seamlessly in an IPv6 network without NAT.

Legenda

• Blue lines: Mesh links between routers.

• Purple directed lines: Upper layer connections. All hosts con open and receive connections each others.

• Documentation addresses conform to RFC 3849 [ref:rfc3849].

Completing the puzzle

While in LiMe L1 isn’t used for packet forwarding, L2 clouds show their limits as they grow oversize and L3 networks feel an unnecessary complication when they shrink too much. LiMe try to take the best from all of them. LiMe developers suggests to extends L2 clouds till the zone keep having homogeneous requirements, so it can be handled with common policies, moreover the probability to suffer a L2 cloud split must remain low enough, another factor to keep in mind is that if a L2 cloud grow too much broadcast packets will start to be significants bandwidth consumers and this is usually unwanted. Typical suggested L2 clouds can spread across areas like a school, a public garden or a small village. L2 clouds doesn’t have mechanisms to connect each other by them self, at this point L3 network start to play his main role, to connect L2 cloud neighborhood each other. While hosts from different cloud can connect each others, as the broadcast domain is split broadcast traffic doesn’t propagate. L3 networks moreover can afford splits and rejoins seamlessly. All of this that can seems some kind of “network magics” is actually done automatically by LiMe and in a quite understandable way thanks to its modular network and software architecture.
Let’s dive into it from the ground up:

L1

At L1 all LiMe nodes of a network, including all clouds pertaining to it, can connect each other if in range, in the case of WiFi Ad Hoc this is obtained by using the same BSSID, in case of cabled Ethernet without further work other then connecting the LiMe nodes to the same switch and in case of WiFi ieee80211s it is obtained using the same mesh ID.

Figure 4. LiMe nodes from different clouds connecting each other at L1.

Legenda

• Black routers: LiMe routers on multiple clouds.

• Blue lines: L1 links between routers.

L1 + L2

Each cloud run L2 routing protocol (by default batman-adv) on top of a different 802.1ad VLAN, so although at L1 different clouds can overlap and connect each other there is no visibility from batman-adv point of view, this creates different broadcast domains also for hosts connected to different clouds.

Figure 5. LiMe nodes from different clouds connecting each other at L1 but not at L2.

Legenda

• Black routers: LiMe routers on multiple clouds.

• Blue lines: L1 links between routers.

• Brown lines: L2 links of cloud 1.

• Orange lines: L2 links of cloud 4.

• Fuchsia lines: L2 links of cloud 9.

L1 + L3

All nodes run the L3 routing protocol (by default BMX6) on the same 802.1ad VLAN, so where there is a L1 link a L3 link exists too. This permits different clouds to automatically connect each other.

Figure 6. LiMe nodes from different clouds connecting each other at L1 and L3.

Legenda

• Black routers: LiMe routers on multiple clouds.

• Blue lines: L1 links between routers.

• Grey lines: L3 links between routers.

L1 + L2 + L3

Joining all pieces together we obtain what usually a community network using LiMe looks like. A bunch of roaming enabled clouds that route internal traffic at L2 and use L3 for communication outside the cloud.

Figure 7. LiMe community network detailing links at L1, L2 and L3.

Legenda

• Black routers: LiMe routers on multiple clouds.

• Blue lines: L1 links between routers.

• Brown lines: L2 links of cloud 1.

• Orange lines: L2 links of cloud 4.

• Fuchsia lines: L2 links of cloud 9.

• Grey lines: L3 links between routers.

L1 + L2 + L3 over L2

A further yet unexplored setup is to automatically enable L3 routing only on nodes that connects multiple cloud and take advantage of the L2 cloud to exchange L3 information instead of use all L1 links, intuitively this could improve scalability but at least a prototype implementation and some empirical data is needed before feasibility could be evaluated.

Figure 8. LiMe community network detailing links at L1, L2 and L3 over L2.

Legenda

• Black routers: LiMe routers on multiple clouds.

• Blue lines: L1 links between routers.

• Brown lines: L2 links of cloud 1.

• Orange lines: L2 links of cloud 4.

• Fuchsia lines: L2 links of cloud 9.

• Grey lines: L3 links between “frontier” routers.

Software Architecture

LiMe meta-firmware is structured as a set of OpenWrt packages, it is licensed under the GNU Affero General Public License v3 and written in Lua programming language in a custom object-oriented style ^[1], the choice of Lua is not a matter of tastes but of convenience, many of OpenWrt libraries are written in Lua or provides binding to it, moreover Lua interpreter fits in few kilobytes [ref:luavspy] that is very important working with embedded devices, while it is particularly suitable for scripting it offer elegant multi-paradigm without making it bloated or unreadable. LiMe joint effort is not the first time multiple communities try to join forces to create a common firmware but the modular structure and the usage of Lua have determined its success adapting to different wireless communities needs without loose code readability, in contrast to previous efforts that where not modular or tended to have mostly unreadable code written for Almquist shell and other ones that tried to use C but ended up writing more code to deal with memory management then for the original objective, LiMe meta-firmware as of today offer more than thirty OpenWrt packages structured and written in a comprehensible manner. LiMe meta-firmware provides a command line tool lime-config that when invoked simply delegate the work of updating the configurations to installed LiMe modules by calling their clean() and configure() methods, LiMe minimal installation already ships necessary modules but the architecture is meant to be extensible so more modules can be added if needed. LiMe modules are often modular themselves, an UML scheme may help the comprehension of LiMe architecture as a whole.

Figure 9. UML diagram of lime-system package.

Module: hardware detection

Hardware detection module is in charge of detecting hardware components of the router and if some of them need a special configuration in order to work according to LiMe needs generate that configuration, to accomplish its task as you cas see in UML diagram of lime-system package. it is modular itself and it’s submodules are called detectors. For each detector installed ( such as lime-hwd-usbradio ) it execute the respective clean() method that is supposed to cleanup outdated configuration and the detect_hardware() method that is supposed to configure the hardware and can write specific config sections in /etc/config/lime to be read by others lime components if necessary.

Module: wireless

Wireless module detect WiFi interfaces and prepare them for usage with LiMe, to accomplish its task it reads and write to /etc/config/lime and /etc/config/wireless, as depicted in UML diagram of lime-system package. it is modular itself and delegate part of the works to his submodules called mode. Each mode must implement a method called setup_radio(radio, args) that take as argument an uci.wireless.radio table and optional adjuntive arguments and is in charge of configure the WiFi radio to operate as needed by LiMe.

Module: network

Network module is in charge of configuring network interfaces and routing protocols, it is a big task to accomplish in a nifty way to do it is modular too. It offers a tight API to his modules that are called proto (such as lime-proto-bgp) while they must expose to network module a statefull interface that consists of:

• configured flag, it is false by default and setted to true when the proto is configured.

• configure(args) this method is in charge of configuring the underlying routing protocol, args is an array containing all parameters found in configuration file, parameters have positional meaning.

• setup_interface(ifname, args) this function is called once per each networking interface found on the router ifname is the Linux name of the interface and args is an array containing all related parameters found in configuration file.

• apply() usually just restart the underlying routing daemon to make the new configurations effective.

Module: firewall

Configure OpenWrt firewall if installed or iptables and ebtables according to LiMe needs.

Module: system

Configure system general settings like host-name and other miscellaneous stuff.

Hardware detector: ground-routing

Ground routing is a node setup similar to Modelo Híbrido in the sense of splitting routing and geographic linking task into different devices. In this setup a device running LiMe is in charge of routing while other devices justs deal with physic and link layer stuff. Those devices can keep their original fabric firmware, or being flashed with a plain OpenWrt image or even run a stripped version of LiMe. The name “ground routing” came from the habit of routing enthusiasts, to keep the L1/L2 devices on which they do not usually change the configurations on the roof (necessary to have good WiFi links) while the “ground router”, which is the subject of their experiments, is kept inside the house, “on the ground”, so if the configuration changes accidentally disrupt connectivity to the router, physical access is simple and there is no need to access the roof for recovery. As this functionality is not needed in default setup lime-hwd-ground-routing is distributed as a separate package and is not included in the default LiMe distribution.

Figure 10. UML diagram of lime-hwd-ground-routing package.

When invoked by lime.hardware_detection, lime.hwd.ground_routing.clean() clean outdated ground routing sections, then lime.hwd.ground_routing.detect_hardware() creates ground routing configuration sections that eventually can include 802.11q interfaces or hardware switch configurations [ref:owrthwswitch][ref:owrtswitch], according to node owner needs and hardware capabilities. In contrast to Modelo Híbrido and Ninix.org Routing a terra that are meant for a specific routing protocol (BGP the first [ref:mdhd], and OLSR the latter [ref:nnxgr]) LiMe Ground Routing is routing protocol agnostic, thanks to the LiMe modular structure it is focused just on preparing the L1/L2 setup on the LiMe router and then the upper layer routing modules treat L2 interfaces created by Ground Routing the same as any other L2 interface [ref:limegr]. This permit to LiMe networks to use any routing protocol on top of a Ground Routing setup and not being limited to a specific routing protocol, thanks to this abstraction LiMe can also “emulate” Ninix.org Routing a terra just installing both lime-hwd-ground-routing and lime-proto-olsr, and Guifi.net Modelo Híbrido just installing both lime-hwd-ground-routing and lime-proto-bgp that is part of the work done during this internship.

Hardware detector: openwrt-wan

This hardware detector is shipped as an optional package lime-hwd-openwrt-wan, but is installed and enabled by the default in LiMe distribution.

Figure 11. UML diagram of lime-hwd-openwrt-wan package.

Low cost consumer hardware, that community networks often use, have multiple Ethernet ports, although from a technical point of view those Ethernet ports are pretty much equivalent each other, vendors usually present them as different to the end user, in particular there is usually a separated port labeled WAN end the other ports usually grouped together labeled LAN [ref:owrtwdr3600]. The end user is told by the router vendor that the WAN port is meant to be connected to the Internet connection and the LAN ports to be connected to local computers. To avoid confusing the user this convention is usually respected both in OpenWrt and in LiMe, although because technically those ports are equivalents there is no automatic way to determine which port is WAN by software. OpenWrt developers when they add support and write the default configuration for a specific device they let the port labeled as WAN to auto-configure via DHCP while they configure as static the ones labeled as LAN by the vendor [ref:owrtlanwan]. LiMe does support almost every device that is well supported by OpenWrt so, instead of duplicating the work of determine if there is a WAN labeled port and which port is it, delegate this task to lime-hwd-openwrt-wan that check which is the WAN port in the OpenWrt default configuration that is backed up into /etc/lime/config.original/ by LiMe at first boot [ref:99limeconfig] and configure it as needed by LiMe and to behave as user expect to.

lime.hwd.openwrt_wan.detect_hardware() snippet that retrieves WAN ifname.

local uci_o = libuci:cursor()
uci_o:set_confdir("/etc/lime/config.original")
uci_o:set_savedir("/dev/null")
local ifname = uci_o:get("network", "wan", "ifname")

Hardware detector: usb-radio

This hardware detector is distributed as an optional package lime-hwd-usbradio, it is in charge of detecting WiFi USB radio.

Figure 12. UML diagram of lime-hwd-usbradio package.

Most consumer routers features one or more USB ports, to setup print servers or share files from an attached disk. With OpenWrt those ports are usually usable as full USB and not only for specific tasks. Taking advantage of this community networks researcher started creating low cost multi-radio routers by plugging USB WiFi radios into those routers. Although OpenWrt configuration system doesn’t recognize those USB radio automatically so the users had to configure them manually. To fill the gap between OpenWrt wireless configuration system and having USB radio automatically working lime-hwd-usbradio has been developed. As OpenWrt is compatible with Hotplug system [ref:owrthotplug] when an USB radio is connected to the router an Hotplug event is fired [ref:hotplug], lime.hwd.usbradio catch that event trough a crafted Hotplug hook [ref:usbradiohook] and configure the USB radio taking in account his capabilities. In the long run the lack of good quality support for adhoc or ieee80211s WiFi modes in Linux drivers for USB radios especially for the cheaper ones has discouraged users to use them for mesh links. Currently USB radios detected by lime-hwd-usbradio are mainly used in AP or STA WiFi modes.

Protocol: adhoc

This protocol is shipped as part of lime-system package as you can see in UML diagram of lime-system package. and it’s enabled by default, it is charge of just generating the network configuration that bring up L2 interfaces corresponding to L1 radio interfaces configured as of LiMe WiFi Mode adhoc.

Protocol: anygw

This protocol is shipped as an optional package lime-proto-anygw, but is installed and enabled by the default in LiMe distribution.

Figure 13. UML diagram of lime-proto-anygw package.

In WiFi networks usually when a client move and get far from the AP it is associated, if it sense better signal from another AP with the same SSID it is expected to automatically disassociate from the far node and associate to the near one. When during this process upper network layers, specifically the user, doesn’t experience a connection interruption it is commonly referred as roaming [ref:wkpdwlan]. LiMe mesh networks does supports roaming at L2. In L2 roaming scenarios, because the clients are usually configured via DHCP and it operates at an higher layer then roaming, the client keep using the same gateway of when it associated first time without caring of it’s position, this behavior may cause client’s packets be routed in a suboptimal way, moreover if the client keep roaming far away from the initial gateway it may experience low network performances.

Figure 14. Roaming without anygw.

Legenda

• Blue dashed lines: Mesh links

• Purple dot dashed line: Client packets routed at L3

• Green dot dashed line: Client packets routed at L2

To avoid this unwanted behavior lime-proto-anygw offer a light implementation of anycast gateway. Anygw builds the virtual Ethernet interface anygw on top of LAN protocol client’s access interface br-lan using the Linux kernel module macvlan [ref:macvlan][ref:linuxswitching]. When anycast gateway is enabled the first address of the mesh network subnet is reserved for anycast gateway operation, together with the IP also a crafted locally administered but valid unicast MAC address [ref:macaddress] is used, all anycast gateways on a LiMe L2 cloud share the same IP and MAC address.

anygw addresses generation.

local ipv4, ipv6 = network.primary_address()

local anygw_mac = "aa:aa:aa:aa:aa:aa"
local anygw_ipv6 = ipv6:minhost()
local anygw_ipv4 = ipv4:minhost()

In this way when a client sends a packets directed outside it’s cloud it gets “intercepted” and optimally routed at L3 by the nearest router.

Figure 15. Roaming with anygw.

Legenda

• Blue dashed lines: Mesh links

• Purple dot dashed line: Client packets routed at L3

• Green dot dashed line: Client packets routed at L2

Protocol: batadv

This protocol is shipped as an optional package lime-proto-batadv, but is installed and enabled by the default in LiMe distribution. It configures batman-adv to create LiMe L2 clouds, and setup 802.1ad VLAN as needed by LiMe setup.

Figure 16. UML diagram of package lime-proto-batadv.

Protocol: bmx6

This protocol is shipped as an optional package lime-proto-bmx6, but is installed and enabled by the default in LiMe distribution. It configures BMX6 for L3 routing between LiMe L3 networks and eventually with upstream connection, it also setup 802.1ad VLAN for BMX6 as needed by LiMe setup.

Figure 17. UML diagram of package lime-proto-bmx6.

Protocol: ieee80211s

This protocol is shipped as part of lime-system package as you can see in UML diagram of lime-system package. but it is not enabled by default, lime.proto.ieee80211s is a relatively new born into LiMe family and the similarly used lime.proto.adhoc is preferred as default configuration because it has been more tested and is well known to community networks participants. Similarly to his older brother lime.proto.adhoc, lime.proto.ieee80211s is charge of just generating the network configuration that bring up L2 interfaces corresponding to L1 radio interfaces configured as of LiMe WiFi Mode ieee80211s.

Protocol: lan

As you can see in UML diagram of lime-system package. lime.proto.lan is shipped as part of lime-system package. LiMe LAN protocol is enabled by default and is meant to configure the network local to the node, in the default configuration it is also in charge of giving access to the mesh network at layer 2 to clients next to the node. It configure the node main IPv4 and IPv6 and bridge all network interfaces meant to give access to clients like wireless AP interfaces and Ethernet interfaces into br-lan.

WiFi mode: adhoc

As you can see in UML diagram of lime-system package. lime.mode.adhoc is shipped as part of lime-system package. LiMe WiFi mode adhoc is enabled by default and configures WiFi interfaces in adhoc mode that are used by upper layer protocols to establish links between LiMe nodes.

WiFi mode: ap

As you can see in UML diagram of lime-system package. lime.mode.ap is shipped as part of lime-system package. LiMe WiFi mode ap is enabled by default and configures WiFi interfaces as an AP. Those interfaces in default configuration are used by upper layer protocols to provide access to clients, but it is possible to use them for creating links too.

WiFi mode: ieee80211s

As you can see in UML diagram of lime-system package. lime.mode.ieee80211s is shipped as part of lime-system package. LiMe WiFi mode ieee80211s is not enabled by default as lime.mode.adhoc is generally preferred because it has been more tested over time. LiMe WiFi mode ieee80211s configures WiFi interfaces in ieee80211s mode but with mesh forwarding disabled, so ieee80211s is used only as link layer and not for packet forwarding that is delegated to upper layer protocols that take advantage of the WiFi interfaces created by lime.mode.ieee80211s to establish links.

Developed tools

lime-system

Even if LiMe is highly modular, to have a sane and modular itself BGP route exchange implementation required some modification to LiMe core, in particular the abstract class Lime Routing Protocol has been extended by adding the bgp_conf(…​) method. A Lime Routing Protocol which want to exchange routes with BGP should implement that method in a way that it does the necessary configuration of the proto itself and returns to the caller a string containing a BIRD snippet configuration for route exchange with that proto.

Figure 18. UML diagram of lime-system package, evidenced in green parts developed during the internship.

The new API call lime.proto.bgp_conf(templateVarsIPv4, templateVarsIPv6) is called by lime.proto.bgp for each lime.proto that has been requested to exchange routes with BGP, this function take as parameters two tables that are both readable to read already defined template variable and writable to eventually define additional template variables to pass back to lime.proto.bgp and returns a template snippet that is appended to the BIRD configuration file by lime.proto.bgp. UML diagram of lime-proto-bgp package, evidenced in green parts developed during the internship. display as an UML diagram the interaction between lime.proto.bgp and lime.proto.*.

Protocol: lan

As lime.proto.lan does very minimal work at L3 it doesn’t need much interaction with BGP to announce his subnet route. Implementing lime.proto.lan.bgp_conf(…​) has been quite simple as you can see in the following code snippet.

function lan.bgp_conf(templateVarsIPv4, templateVarsIPv6)
	local base_conf = [[
protocol direct {
	interface "br-lan";
}
]]
	return base_conf
end

Protocol: anygw

As the interaction of lime.proto.anygw with lime.proto.bgp is very similar to the one of lime.proto.lan. The modifications necessaries have been very minimal and limited to implementing lime.proto.anygw.bgp_conf(…​) as you can see in the following code snippet, and in the UML diagram.

function anygw.bgp_conf(templateVarsIPv4, templateVarsIPv6)
	local base_conf = [[
protocol direct {
	interface "anygw";
}
]]
	return base_conf
end

Figure 19. UML diagram of lime-proto-anygw package, evidenced in green parts developed during the internship.

Protocol: bgp

To accomplish the task of interoperability between Libre-Mesh networks and Guifi.net BGP infrastructure, I have created lime.proto.bgp that implements the abstract class LiMe Routing Protocol and is shipped as the optional package lime-proto-bgp.

Figure 20. UML diagram of lime-proto-bgp package, evidenced in green parts developed during the internship.

This new LiMe Routing Protocol is in charge of configuring the LiMe system so it is capable of BGP routing using BIRD as back-end. To interconnect BGP zones with LiMe zones it establish peering with known BGP peer and export route learned via peering to the main kernel table. To instruct the BGP zones on how to reach the LiMe zones learning from main kernel table is enabled too, the route learned from the main kernel table are then exported to the BGP peering.

Snippet of lime-proto-bgp code showing how routes are learned and exported from and to main kernel table.

local base_template = [[
protocol kernel {
	learn;
	scan time 20;
	export all;
}
]]

Mesh routing protocols do usually have metrics to describe the quality of a path to a route, but BGP has not a similar concept, although it is possible to associate multiples attributes to BGP routes those aren’t treated in an uniform way by different vendors implementations and in eBGP are often discarded or ignored [ref:bgpattributediscard]. Because there is no reliable way to convert mesh quality of path to a comparable BGP attribute, I decided to take a conservative approach and try to give to BGP zones the false but always worse then reality information that all mesh paths are to avoid if possible. The basic idea is that if a pure BGP path exists it should be preferred to a path including mesh networks because BGP can at least count hops on a pure BGP network but cannot understand if a mesh path is good or maybe the worse path possible. In eBGP setups (like Guifi.net) the most reliable way to share a route with low preference is to artificially enlarge it’s AS path, this technique is called AS-path-prepending [ref:bgpaspathprepending] and it’s of common usages in situation where someone want to share a route but artificially lowering the preference.

Snippet of lime-proto-bgp code showing how mesh paths are avoided if possible.

local meshPenalty = args[4] or 8

local mp = "bgp_path.prepend("..localAS..");\n"
for i=1,meshPenalty do
	mp = mp .. "\t\t\tbgp_path.prepend("..localAS..");\n"
end

As in BGP direct interaction with network devices is almost absent it doesn’t need any configuration for each interface so lime.proto.bgp.setup_interface(ifname, args) is an empty method, this may seems a cheap simplification but come with a not so cheap hidden cost, BGP doesn’t offer any mechanism of node discovering so for each BGP peer peering configuration must be generated by LiMe.

Snippet of lime-proto-bgp code showing how peering configuration is generated for each BGP peer.

local peer_template = [[
protocol bgp {
	import filter fromBgp;
	export filter toBgp;

	local as $localAS;
	neighbor $remoteIP as $remoteAS;
}
]]

local function apply_peer_template(s)
	s.localAS = localAS
	if string.find(s.remoteIP, ":", 1, true) then
		bird6_config = bird6_config .. utils.expandVars(peer_template, s)
	elseif string.find(s.remoteIP, ".", 1, true) then
		bird4_config = bird4_config .. utils.expandVars(peer_template, s)
	end
end
config.foreach("bgp_peer", apply_peer_template)

While lime.proto.bgp is directly responsible of configuring BGP routing specific parts like BGP peering and BIRD filters, it delegates through the LiMe modularization pattern to the others LiMe Routing Protocol the specific configuration needed to make possible to automatize the inter-operation between each specific lime.proto and BGP, this has been accomplished calling for each of them the new introduced callback lime.proto.bgp_conf(…​) and collecting the returned snippets into BIRD configuration file.

Protocol: bmx6

This package was already in charge of configuring BMX6 for L3 routing, now it also generate configuration for routes exchange between BGP and BMX6 networks, although it is not much code it has required time for investigation and for experimental tuning. The problems discovered during the experimental phase wasn’t intrinsic of this module code, but were mainly BMX6 performance bottlenecks exposed by the configuration generated by this module.

Figure 21. UML diagram of lime-proto-bmx6 package, evidenced in green parts developed during the internship.

While BIRD has quite powerful route filters based on a C like language [ref:birdfilters], in BMX6 redistTable plugin is the mechanism of choice to learn routes. Some filter capability were already present in redistTable but those were not enough for interoperability with BGP. In collaboration with BMX6 developer the redistTable plugin has been extend to be capable of filtering routes by Linux kernel proto number [ref:bmxgit:1dcb463]-[ref:bmxgit:ca9c380]. BMX6 routing protocols associate a metric to describe the quality of a path to a route, BGP has not a similar concept it just know hop count, because there is no reliable way to convert BGP hop count to a comparable BMX6 metric, I have decided to take a conservative approach and try to give to BMX6 zones the false but always worse then reality information that all BGP paths are to avoid if possible. The basic idea is that if a pure BMX6 path exists it should be preferred to a path including BGP networks because BMX6 know the quality of a mesh path while it doesn’t have enough information to know if a BGP path is good or maybe the worse path possible. To accomplish this self imposed requirement redistTable is instructed to announce learned routes with low bandwidth.

Importing BGP routes from a kernel table and announce them as low bandwidth.

-- Enable Routing Table Redistribution plugin
uci:set("bmx6", "table", "plugin")
uci:set("bmx6", "table", "plugin", "bmx6_table.so")

-- Redistribute proto bird routes
uci:set("bmx6", "fromBird", "redistTable")
uci:set("bmx6", "fromBird", "redistTable", "fromBird")
uci:set("bmx6", "fromBird", "table", "200")
uci:set("bmx6", "fromBird", "bandwidth", "100")
uci:set("bmx6", "fromBird", "proto", "12")

Using only words to describe the overall architecture resulting from the developed code would be long and of scarce effectiveness, in Flow of routing information between components. the active components and the flow of routing information are displayed in a simplified manner all together.

Figure 22. Flow of routing information between components.

Legenda

• Black arrows: Routing information flowing inside components of the same process.

• Green arrows: Routing information flowing between kernel routing table and processes via Netlink.

• Yellow arrows: Routing information flowing between kernel and process via socket API.

• Cyan arrows: Routing information flowing between different routers via IP.

• Documentation addresses conform to RFC 5737 [ref:rfc5737].

Experimental results

Interoperability use-cases

Due to LiMe modular nature this development has many possible application that spread out of boundaries of this internship, in the following sections a brief analysis of more common use-cases on which the development and experimentation has been focused is exposed.

Mesh network as distribution

In this scenario a BGP backbone is used as transport network while the mesh network is used as last mile distribution.

Figure 23. Simple exchange topology of a LiMe network used as last mile distribution.

Legenda

• Black and red routers: Routers running LiMe doing routing exchange between BGP and BMX6

• Black routers: Routers running LiMe with pure mesh setup.

• Red routers: Guifi.net routers configured as of Modelo Híbrido.

• Blue straight lines: Cabled mesh links.

• Blue curvy dashed lines: WiFi mesh links.

• Yellow straight lines: Guifi.net links as of Modelo Híbrido.

This use case is someway workable also with the pasts solution but with strong drawbacks:

• Instead of the “red and black” router we would need two devices one of the “red type” let’s call it R and one of the “black type” lets call it B connected, this mean a frontier node cost the double on average.

• The B router would announce statically the whole 10.0.0.0/8 to the mesh network and have R as gateway for the entire 10.0.0.0/8 net, this causes packets generated into the mesh directed to nonexistent hosts in that range will flow all across the mesh and even get into the BGP backbone, effectively reducing bandwidth available for legitimate packets.

• In the R router all the subnets used by the mesh network must be statically routed to B router and then written into the BGP daemon configuration to be announced to the rest of the “red network”, so each time a subnet is added to the the mesh network the R router need to be reconfigured manually, consuming time of some skilled Guifi.net operator.

All those limitations are solved using the software developed during this internship, moreover other more complex and even more common interoperability scenarios gets solved.

Mesh network with multiple exchanges

This scenario is very common in high population density zones, small neighborhood community mesh network grow beyond the district boundaries gaining geographical proximity to multiple backbone nodes, as the number of user usually grow too as the network expand having just an exchange point become a bottleneck, in this case having multiple exchange point can improve the access bandwidth to upstream networks.

Figure 24. Multiple exchange between classic Guifi.net infrastructure and LiMe network example topology.

This scenario in not practically manageable with past technology, the more exchange points are added the trickier it gets to manually write a configuration that works stably. During the internship an automatic solution has been developed although it hasn’t been trivial, the usual Guifi.net eBGP setup is not enough as it causes routing loop [ref:routeloop] in this situation. When an host like 2001:DB8:9::4 from the LiMe network send a packet to an host like 2001:DB8:1::3 into BGP network the packet make his way to the destination without problems. Unfortunately is not the same when 2001:DB8:1::3 send a packet to his friend 2001:DB8:9::4. The packet is correctly routed toward the exchange BGPx2 but there because routes learned via BGP are preferred it’s sent back to BGP4 expecting it have a better path for those packets. BGP4 again thinks that BGPx2 is the best next hop to reach 2001:DB8:9::4 and forward the packet to BGPx2 and so on till the packet TTL is exhausted and one of the routers discard it.

Figure 25. Routing loop discarding packets in naive eBGP multiple exchange setup.

Legenda

• Directed black dashed line: packets flowing in the direction of the arrow.

But what in detail causes this unwanted behavior? We can find the answer is in BGPx2 routing table of which the relevant snippet is reported in the following lines.

2001:DB8:1::/64 via $BGP4 proto bird metric 0
2001:DB8:1::/64 dev bmxOut_BGPx1 proto static metric 1024
2001:DB8:9::/64 via 2001:DB8:8::1 proto bird metric 0
2001:DB8:9::/64 dev bmxOut_BMX1 proto static metric 1024

We can observe there are two routes towards 2001:DB8:9::/64 one learned via BGP (the proto bird ones) that have lower metric and one learned by BMX6 (the ones associated to dev bmx*) that have higher metric, thus the BIRD routes are always preferred. One could think that the problem could be fixed just by giving to BMX6 more preference but this tentative would just fail because it solves the loop in the BGP→BMX6 direction but it creates the exact opposite loop in BMX6→BGP direction. The classic way to solve this problem on BGP networks usually is to adopt the iBGP setup. In a iBGP setup routers share the same ASN so the exchanges can recognize internal routes when they learn them form other eBGP routers and avoid re-adding them to the kernel routing table. To adopt iBGP solve the problem but with an unbearable cost for an auto-organized mesh network that is that each iBGP router must be configured to establish peering connections to each other router in a setup called full-mesh where every router speaks to all other routers directly [ref:wkpdbgp]. At this point, no other solutions then iBGP has been found. I started investigating how iBGP works internally and I discovered that the full-mesh peering is needed because every iBGP router need share the same routing information with all other iBGP routers [ref:ebgpibgp]. In our case as BMX6 is a proactive distance-vector routing daemon it already spreads all routes to all routers so I supposed that in our case the iBGP full-mesh setup may be avoided, and empirical tests proved this supposition true. In fact loop is avoided and every host of both BGP and BMX6 networks can successfully reach every host of the whole Guifi.net (both BMX6 and BGP parts).

Relevant snippet of the resulting routing table using the “smart” iBGP-like setup.

2001:DB8:1::/64 via $BGP4 proto bird metric 0
2001:DB8:1::/64 dev bmxOut_BGPx1 proto static metric 1024
2001:DB8:9::/64 dev bmxOut_BMX1 proto static metric 1024

Mesh network as transport

The most important technical factor that slowed down LiMe adoption into Guifi.net is that the majority of preexistent network infrastructure is built following the Modelo Híbrido so when someone decide to extend Guifi.net infrastructure need to be compatible with Modelo Híbrido, without the development done during this internship the only possible interoperability setup was having the mesh as a leaf distribution network, while now it is possible to build Guifi.net backbone with LiMe being compatible both with mesh and BGP networks, it is even possible to use LiMe to fill the gap between isolated BGP network islands.

Figure 26. Mesh used as transport between two isolated BGP networks.

This will certain encourage LiMe adoption and maybe finally obsolete Modelo Híbrido, giving the possibility to Guifi.net to jump into another phase of horizontal growth with less dependency from manual configuration work, skilled people will have time to work on more interesting and not repetitive configuration tasks, with consequent work quality improvement, costs reduction and increased network accessibility for the masses.

Solving the route count limitation

BMX6 store distributed routes in a shared structure called “node description” for performance reasons the size of this structure is limited and it’s not enough to contain the whole Guifi.net BGP routing table, a first approach to solve this problem was to enable route aggregation in BMX6 redistTable plug-in, but despite the high CPU usage the size of the table was reduced only by half and that was not enough to fit inside a “node description”, moreover the high CPU load caused timeouts in communication between BMX6 and Linux kernel causing BMX6 to crash. To solve this problem in the new version of BMX6 the “node description” has been restructured like a linked list so the node description can contain a pointer to another structure if just one is not enough to store all the needed informations, the only drawback is that this is not compatible with past versions of BMX6 but it doesn’t represents a blocking problem because BMX6 users are used to incompatible changes between versions, this solution has been discussed at BattlemeshV8 [ref:battlemesh] with BMX6 developer which has implemented it and is now in production at our offices in Barcelona.

Reducing the performance footprint

BMX6 redistTable as it was before this internship was not suitable to our needs in fact because it was not ready to handle a big routing table it was using 100% CPU all the time. The high CPU load do not only was stressing our little routers but causing them to have high response time and that was interpreted by other routers as if the link was performing really bad, this caused routers loose connectivity each others. In those condition to further analyze performances wasn’t possible. After some performances fixes in upstream BMX6 code I have been able to see BMX6 running stably. So after setup the performance testbed Performance analysis testbed. with multiple connections to both BGP based portion of Guifi.net and Libre-Mesh based portion of Guifi.net I have proceeded optimizing performance tweaking BMX6 configurations.

Figure 27. Performance analysis testbed.

Legenda

• Black routers: Routers running LiMe with pure mesh setup.

• Black and red routers: Routers running LiMe doing routing exchange between BGP and BMX6

• Red routers: Guifi.net routers configured as of Modelo Híbrido.

• Blue straight lines: Cabled mesh links.

• Blue curvy dashed lines: WiFi mesh links.

• Yellow straight lines: Guifi.net links as of Modelo Híbrido.

• Server and smart-phone pinging each other.

Default configuration

Even if BMX6 run stably its performance impact in the default configuration is not acceptable, as we can observe in Performance graph of default configuration. the system load on BGP↔BMX6 exchanges too often exceed 1, and when it is under it still use more then half system resources living too few CPU time for other tasks such as packet forwarding.

Figure 28. Performance graph of default configuration.

Route aggregation

By default redistTable do routes aggregation that is a CPU and memory intensive work. In our case as Guifi.net has thousands of routes it may have a big impact on the performances, disabling route aggregation is done directly into lime-proto-bmx6 code.

uci:set("bmx6", "fromBird", "aggregatePrefixLen", "128")

As we can see in Performance graph without route aggregation. disabling route aggregation has a sensible positive impact on performances, system load is consistently reduced both on BGPx1 and BGPx2, while a non critical slight increase in memory usage is noticeable on BMX 3. Due to disabled aggregation BMX 3 receive same amounts of route announcements of BGP↔BMX6 exchanges so it’s memory usage align to them.

Figure 29. Performance graph without route aggregation.

Proactive tunnels

By default BMX6 react to every single change in the routing table. As Guifi.net is a quite big network lot of route changes get propagated and sometime some change is reverted before the change itself get propagated to the whole network. This two conditions together were caught causing lot of recalculation and propagation also of temporary changes, if a route was deleted by BIRD and then re-added in a fraction of a second this caused BMX6 recalculating announcements and propagating both changes. To avoid it more options to regulate the BMX6 behavior has been added directly in BMX6 code [ref:bmxgit:52cfaad][ref:bmxgit:5e0d744] and then configured by lime-proto-bmx6 only in cases where default configuration wasn’t appropriated to Guifi.net needs.

uci:set("bmx6", "general", "proactiveTunRoutes", "0")

While this behavior depends on network conditions too, in Performance graph with route aggregation and proactive tunnels both disabled. we can observe a fair system load reduction and a more noticeable reduction in system load oscillation due to solved hypersensitivity to temporary changes.

Figure 30. Performance graph with route aggregation and proactive tunnels both disabled.

Link Signature

Another aspect that may impact performances are the trusted networking features of BMX6 based on cryptography, those features, as examples, permits to filter out spoofed routing informations or to prefer paths made only of trusted nodes [ref:bmxhotmesh][ref:bmxcryptopresentation]. While those features are enable by default there was no option to disable them and Guifi.net LiMe setup doesn’t take advantage of them so an option to permit to tweak them has been introduced in BMX6 [ref:bmxgit:2156de7], and disabled in lime-proto-bmx6 code.

uci:set("bmx6", "general", "linkSignatureLen", "0")

As we can observe in Performance graph with route aggregation, proactive tunnels and link signature all disabled. this change caused a consistent CPU usage reduction, lowering system load to usual values even if the routing table is quite big.

Figure 31. Performance graph with route aggregation, proactive tunnels and link signature all disabled.

Conclusions and future developments

During this internship, seamless interoperability between classic infrastructure and Libre-Mesh networks in Guifi.net has been achieved with satisfactory results. For Guifi.net, that at time of writing counts more then 10 years of activity and nearly 30000 active nodes, this potentially opens new horizons of sustainable growth. Building networks inside Guifi.net with Libre-Mesh doesn’t mean anymore to be relegated to last mile distribution, but to build at same time affordable and auto-configuring Guifi.net carrier networks and doing it with completely free as in freedom software [ref:freesoftware]. The possibility of using free software have big impact, as an example till today Guifi.net has experienced difficulty even to support IPv6 on carrier links because of proprietary routers requiring costly licenses upgrade to enable IPv6 support on their software, or alternatively to completely renew the infrastructure resulting in an awkward situation well known as collective vendor lock-in. This will never happen with free as in freedom routers, but this fact doesn’t mean only IPv6 support. Being a free software solution, combined with the fact that LiMe architecture is modular guarantees an high level of future-proofness. Community participants can eventually extend and modify the software and improve it to satisfy unforeseen needs without high costs. LiMe networks requires significantly less human intervention then Modelo Híbrido based networks, as node installation will require less work, lot of skilled Guifi.net enthusiasts and professionals will have more time to install more nodes or to do more non automatable work like research, development and community networks promotion, this will probably foster even more Guifi.net growth. LiMe decentralized network model does allow more independence from centralized configuration provisioning tools, that are in contrast with the idea of a non-hierarchic collaborative network managed directly by its users, like Guifi.net try to be. This works has also benefited other projects aside of LiMe and Guifi.net, multiple BMX6 bugs and performance bottlenecks have been discovered during the experimental phase and isolated in collaboration with BMX6 main developer Axel Neumann who has fixed them in upstream code [ref:bmxgit:1dcb463]-[ref:bmxgit:ca9c380] [ref:bmxgit:52cfaad] [ref:bmxgit:5e0d744] [ref:bmxgit:2156de7] [ref:bmxgit:170a847]-[ref:bmxgit:43873b8]. Mesh networks in general now benefits of a maturer software stressfully tested with real world huge routing table. Last but not least interoperability between BGP and BMX6 have been achieved in a very modular way, it doesn’t really depends on specific features of the second protocol, so future implementations of route exchange with other protocols like Babel or OLSRv2 will require minimal effort and will benefit of same loop avoidance guarantees explored during this work, making easy to handle even more complex exchange topologies ^[2], with multiple different protocols and multiple independents peers.

Appendix: Code

All the code code written for Libre-Mesh has been published on its public repository [ref:limerepo] in references you can find the list of specific commits [ref:limegit:8291374]-[ref:limegit:241f15a]. All needed modification of BMX6 are already published on its public repository [ref:bmx6repo], in references you can find the list of specific commits [ref:bmxgit:1dcb463]-[ref:bmxgit:ca9c380] [ref:bmxgit:52cfaad] [ref:bmxgit:5e0d744] [ref:bmxgit:2156de7] [ref:bmxgit:170a847]-[ref:bmxgit:43873b8] and on OpendWrt routing feed [ref:owrtrouting] in specific commits [ref:owrtrouting:42d1982]-[ref:owrtrouting:2ebaa92]. The following reported source code is from the files changed into Libre-Mesh packages.

role=include

Appendix: License

© copyright 2015 Gioacchino Mazzurco.
This work is licensed under the Creative Commons Attribution-ShareAlike 4.0 International License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/4.0/.

Glossary

role=include

References

role=include

---

1. In this custom object-oriented style objects are Lua tables. Object’s members (attributes and operations) are table entry with the name of the member as key in the table. Objects may implement some abstract class. Abstract classes aren’t written in code but specified in documentation. There is no check of conformance with abstract class specification.

2. In respect to the ones analyzed in the scope of this work, see Experimental results for more information.