注 :你 不应该 为了加快处理您的过滤列表而使用正则表达 式。你可能听过类似的说法,但其实那已经过时了。从 Adblock Plus 0.7 开始,基本过滤规则的处理已经比正规表达式快了。
元素隐藏
基本规则
有时你可能会发现无法阻挡某些内嵌在网页中的文字广告。如果查看源码的话,可能发现类似这样的代码:
<divclass="textad">
Cheapest tofu, only here and now!
<div><divid="sponsorad">
Really cheap tofu, click here!
<div><textad>
Only here you get the best tofu!
</textad>
I used to have a static IP address on my home ADSL connection, but then I moved to BT Infinity, and they don’t provide that ability. For whatever reason, my Infinity connection resets a few times a week, and it always results in a new IP address.
Since I wanted to be able to connect to a service on my home IP address, I signed up to dyn.com and used their free service for a while, using a CNAME with my hosting provider (Gandi) so that I could use a single common host, in my own domain, and point it to the dynamic IP host and hence, dynamic IP address.
While this works fine, I’ve had a few e-mails from dyn.com where either the update process hasn’t been enough to prevent the ’30 day account closure’ process, or in recent times, a mail saying they’re changing that and you now need to log in on the website once every 30 days to keep your account.
I finally decided that since I run a couple of VPSs, and have good control over DNS via Gandi, I may as well run my own bind9 service and use the dynamic update feature to handle my own dynamic DNS needs. Side note: I think Gandi do support DNS changes through their API, but I couldn’t get it working. Also, I wanted something agnostic of my hosting provider in case I ever move DNS in future (I’m not planning to, since I like Gandi very much).
The basic elements of this are,
a bind9 service running somewhere, which can host the domain and accept the updates.
delegation of a subdomain to that bind9 service. Since Gandi runs my top level domain for me, I need to create a subdomain and delegate to it, and then make dynamic updates into that subdomain. I can still use CNAMEs in the top level domain to hide the subdomain if I wish.
configuration of the bind9 service to accept secure updates.
a script to do the updates.
In the interests of not re-inventing the wheel, I copied most of the activity from this post. But I’ll summarise it here in case that ever goes away.
Installing / Configuring bind9
You’ll need somewhere to run a DNS (bind9 in my case) service. This can’t be on the machine with the dynamic IP address for obvious reasons. If you already have a DNS service somewhere, you can use that, but for me, I installed it on one of my Debian VPS machines. This is of course trivial with Debian (I don’t use sudo, so you’ll need to be running as root to execute these commands),
apt-get install bind9 bind9-doc
If the machine you’ve installed bind9 onto has a firewall, don’t forget to open ports 53 (both TCP and UDP). You now need to choose and configure your subdomain. You’ll be creating a single zone, and allowing dynamic updates.
The default config for bind9 on Debian is in /etc/bind, and that includes zone files. However, dynamically updated zones need a journal file, and need to be modified by bind. I didn’t even bother trying to put the file into /etc/bind, on the assumption bind won’t have write access, so instead, for dynamic zones, I decided to create them in /var/lib/bind. I avoided /var/cache/bind because the cache directory, in theory, is for transient files that applications can recreate. Since bind can’t recreate the zone file entirely, it’s not appropriate to store it there.
I added this section to /etc/bind/named.conf.local,
// Dynamic zone
zone "home.example.com" {
type master;
file "/var/lib/bind/home.example.com";
update-policy {
// allow host to update themselves with a key having their own name
grant *.home.example.com self home.example.com.;
};
};
This sets up the basic entry for the master zone on this DNS server.
Create Keys
So I’ll be honest, I’m following this section mostly by rote from the article I linked. I’m pretty sure I understand it, but just so you know. There are a few ways of trusting dynamic updates, but since you’ll likely be making them from a host with a changing IP address, the best way is to use a shared secret. That secret is then held on the server and used by the client to identify itself. The configuration above allows hosts in the subdomain to update their own entry, if they have a key (shared secret) that matches the one on the server. This stage creates those keys.
This command creates two files. One will be the server copy of the key file, and can contain multiple keys, the other will be a single file named after the host that we’re going to be updating, and needs to be moved to the host itself, for later use.
ddns-confgen -r /dev/urandom -q -a hmac-md5 -k thehost.home.example.com -s thehost.home.example.com. | tee -a /etc/bind/home.example.com.keys > /etc/bind/key.thehost.home.example.com
The files will both have the same content, and will look something like this,
You should move the file key.thehost.home.example.com to the host which is going to be doing the updating. You should also change the permissions on the home.example.com.keys file,
You should now return to /etc/bind/named.conf.local and add this section (to use the new key you have created),
// DDNS keys
include "/etc/bind/home.example.com.keys";
With all that done, you’re ready to create the empty zone.
Creating the empty Zone
The content of the zone file will vary, depending on what exactly you’re trying to achieve. But this is the one I’m using. This is created in /var/lib/bind/home.example.com,
In this case, namesever.example.com is the hostname of the server you’ve installed bind9 onto. Unless you’re very careful, you shouldn’t add any static entries to this zone, because it’s always possible they’ll get overwritten, although of course, there’s no technical reason to prevent it.
At this stage, you can recycle the bind9 instance (/etc/init.d/bind9 reload), and resolve any issues (I had plenty, thanks to terrible typos and a bad memory).
Delegation
You can now test your nameserver to make sure it responds to queries about the home.example.com domain. In order to properly integrate it though, you’ll need to delegate the zone to it, from the nameserver which handles example.com. With Gandi, this was as simple as adding the necessary NS entry to the top level zone. Obviously, I only have a single DNS server handling this dynamic zone, and that’s a risk, so you’ll need to set up some secondaries, but that’s outside the scope of this post. Once you’ve done the delegation, you can try doing lookups from anywhere on the Internet, to ensure you can get (for example) the SOA for home.example.com.
Making Updates
You’re now able to update the target nameserver, from your source host using the nsupdate command. By telling it where your key is (-k filename), and then passing it commands you can make changes to the zone. I’m using exactly the same format presented in the original article I linked above.
cat <<EOF | nsupdate -k /path/to/key.thehost.home.example.com
server nameserver.example.com
zone home.example.com.
update delete thehost.home.example.com.
update add thehost.home.example.com. 60 A 192.168.0.1
update add thehost.home.example.com. 60 TXT "Updated on $(date)"
send
EOF
Obviously, you can change the TTL’s to something other than 60 if you prefer.
Automating Updates
The last stage, is automating updates, so that when your local IP address changes, you can update the relevant DNS server. There are a myriad ways of doing this. I’ve opted for a simple shell script which I’ll run every couple of minutes via cron, and have it check and update DNS if required. In my instance, my public IP address is behind a NAT router, so I can’t just look at a local interface, and so I’m using dig to get my IP address from the opendns service.
This is my first stab at the script, and it’s absolutely a work in progress (it’s too noisy at the moment for example),
#!/bin/sh
# set some variables
host=thehost
zone=home.example.com
dnsserver=nameserver.example.com
keyfile=/home/bob/conf/key.$host.$zone
#
# get current external address
ext_ip=`dig +short @resolver1.opendns.com myip.opendns.com`
# get last ip address from the DNS server
last_ip=`dig +short @$dnsserver $host.$zone`
if [ ! -z “$ext_ip” ]; then
if [ ! -z “$last_ip” ]; then
if [ “$ext_ip” != “$last_ip” ]; then
echo “IP addresses do not match (external=$ext_ip, last=$last_ip), sending an update”
It’s no secret to anyone, APKs out there are getting bigger and bigger. While simple/single-task apps were 2MB at the time of the first versions of Android, it is now very common to download 10 to 20MB apps. The explosion of APK file size is a direct consequence of both users expectations and developers experience acquisition. Several reasons explain this dramatic file size increase:
The multiplication of dpi categories ([l|m|tv|h|x|xx|xxx]dpi)
The evolution of the Android platform, development tools and the libraries ecosystem
The ever-increasing users’ expectations regarding high quality UIs
etc.
Publishing light-weight applications on the Play Store is a good practice every developer should focus on when designing an application. Why? First, because it is synonymous with a simple, maintainable and future-proof code base. Secondly, because developers would generally prefer staying below the Play Store current 50MB APK limit rather than dealing with download extensions files. Finally because we live in a world of constraints: limited bandwidth, limited disk space, etc. The smaller the APK, the faster the download, the faster the installation, the lesser the frustration and, most importantly, the better the ratings.
In many (not to say all) cases, the size growth is mandatory in order to fulfill the customer requirements and expectations. However, I am convinced the weight of an APK, in general, grows at a faster pace than users expectations. As a matter of fact, I believe most apps on the Play Store weight twice or more the size they could and should. In this article, I would like to discuss about some techniques/rules you can use/follow to reduce the file size of your APKs making both your co-workers and users happy.
The APK file format
Prior to looking at some cool ways to reduce the size of our apps, it is mandatory to first understand the actual APK file format. Put simply, an APK is an archive file containing several files in a compressed fashion. As a developer, you can easily look at the content of an APK just by unzipping it with the unzip command. Here is what you usually get when executing unzip <your_apk_name>.apk1:
Most of the directories and files shown above should look familiar to developers. They mostly reflect the project structure observed during the design & development process: /assets, /lib, /res, AndroidManifest.xml. Some others are quite exotic at first sight. In practice, classes.dex, contains the dex compiled version of you Java code while resources.arsc includes precompiled resources e.g. binary XML (values, XML drawables, etc.).
Because an APK is a simple archive file, it means it has two different sizes: the compressed file size and the uncompressed one. While both sizes are important, I will mainly focus on the compressed size in this article. In fact, a great rule of thumb is to consider the size of the uncompressed version to be proportional to the archive: the smaller the APK, the smaller the uncompressed version.
Reducing APK file size
Reducing the file size of an APK can be done with several techniques. Because each app is different, there is no absolute rule to put an APK on diet. Nevertheless, an APK consists of 3 significant components we can easily act on:
Java source code
resources/assets
native code
The tips and tricks below all consist on minimizing the amount of space used per component reducing the overall APK size in the process.
Have a good coding hygiene
It probably seems obvious but having a good coding hygiene is the first step to reducing the size of your APKs. Know your code like the back of one’s hand. Get rid of all unused dependency libraries. Make it better day after day. Clean it continuously. Focusing on keeping a clean and up-to-date code base is generally a great way to produce small APKs that only contain what is strictly essential to the app.
Maintaining an unpolluted code base is generally easier when starting a project from scratch. The older the project is, the harder it is. As a fact, projects with a large historical background usually have to deal with dead and/or almost useless code snippets. Fortunately some development tools are here to help you do the laundry…
Run Proguard
Proguard is an extremely powerful tool that obfuscates, optimizes and shrinks your code at compile time. One of its main feature for reducing APKs size is tree-shaking. Proguard basically goes through your all of your code paths to detect the snippets of code that are unused. All the unreached (i.e. unnecessary) code is then stripped out from the final APK, potentially radically reducing its size. Proguard also renames your fields, classes and interfaces making the code as light-weight as possible.
As you may have understood, Proguard is extremely helpful and efficient. But with great responsibilities comes great consequences. A lot of developers consider Proguard as an annoying development tool because, by default, it breaks apps heavily relying on reflection. It’s up to developers to configure Proguard to tell it which classes, fields, etc. can be processed or not.
Use Lint extensively
Proguard works on the Java side. Unfortunately, it doesn’t work on the resources side. As a consequence, if an image my_image in res/drawable is not used, Proguard only strips it’s reference in the R class but keeps the associated image in place.
Lint is a static code analyzer that helps you to detect all unused resources with a simple call to ./gradlew lint. It generates an HTML-report and gives you the exhaustive list of resources that look unused under the “UnusedResources: Unused resources” section. It is safe to remove these resources as long as you don’t access them through reflection in your code.
Lint analyzes resources (i.e. files under the /res directory) but skips assets (i.e. files under the /assets directory). Indeed, assets are accessed through their name rather than a Java or XML reference. As a consequence, Lint cannot determine whether or not an asset is used in the project. It is up to the developer to keep the /assets folder clean and free of unused files.
Be opinionated about resources
Android supports a very large set of devices at its core. In fact, Android has been designed to support devices regardless of their configuration: screen density, screen shape, screen size, etc. As of Android 4.4, the framework natively supports various densities: ldpi, mdpi, tvdpi, hdpi, xhdpi, xxhdpi and xxxhdpi. Android supporting all these densities doesn’t mean you have to export your assets in each one of them.
Don’t be afraid of not bundling some densities into your application if you know they will be used by a small amount of people. I personally only support hdpi, xhdpi and xxhdpi2 in my apps. This is not an issue for devices with other densities because Android automatically computes missing resources by scaling an existing resource.
The main point behind my hdpi/xhdpi/xxhdpi rule is simple. First, I cover more than 80% of my users. Secondly xxxhdpi exists just to make Android future-proof but the future is not now (even if it’s coming very quickly…). Finally I actually don’t care about the crappy/low-res densities such as mdpi or ldpi. No matter how hard I work on these densities, the result will look as horrible as letting Android scaling down the hdpi variant.
On a same note, having a single variant of an image in drawable-nodpi also can save you space. You can afford to do that if you don’t think scaling artifacts are outrageous or if the image is displayed very rarely throughout the app on day-to-day basis.
Minimize resources configurations
Android development often relies on the use of external libraries such as Android Support Library, Google Play Services, Facebook SDK, etc. All of theses libraries comes with resources that are not necessary useful to your application. For instance, Google Play Services comes with translations for languages your own application don’t even support. It also bundles mdpi resources I don’t want to support in my application.
Starting Android Gradle Plugin 0.7, you can pass information about the configurations your application deals with to the build system. This is done thanks to the resConfig and resConfigs flavor and default config option. The DSL below prevents aapt from packaging resources that don’t match the app managed resources configurations:
Aapt comes with a lossless image compression algorithm. For instance, a true-color PNG that does not require more than 256 colors may be converted to an 8-bit PNG with a color palette. While it may reduce the size of your resources, it shouldn’t prevent you from embracing the lossy PNG preprocessor optimization path. A quick Google search yields several tools such as pngquant, ImageAlpha or ImageOptim. Just pick the one that best fits your designer workflow and requirements and use it!
A special type of Android-only images can also be minimized: 9-patches. As far as I know, no tools have been specifically created for this. However, this can be done fairly easily just by asking your designer to reduce the stretchable and content areas to a minimum. In addition to optimizing the asset weight, it will also make the assets maintenance way easier in the long term.
Limit the number of architectures
Android is generally about Java but there are some rare cases where applications need to rely on some native code. Just like you should be opinionated about resources, you should too when it comes to native code. Sticking to armabi and x86 architecture is usually enough in the current Android eco-system. Here is an excellent article about native libraries weight reduction.
Reuse whenever possible
Reusing stuff is probably one of the first important optimization you learn when starting developing on mobile. In a ListView or a RecyclerView, reusing helps you keep a smooth scrolling performance. But reusing can also help you reduce the final size of your APK. For instance, Android provides several utilities to re-color an asset either using the new android:tint and android:tintMode on Android L or the good old ColorFilter on all versions.
You can also prevent packaging resources that are only a rotated equivalent of another resource. Let’s say you have 2 images named ic_arrow_expand and ic_arrow_collapse :
You can easily get rid of ic_arrow_collapse by creating a RotateDrawable relying on ic_arrow_expand. This technique also reduces the amount of time your designer requires to maintain and export the collapsed asset variant:
In some cases rendering graphics directly for the Java code can have a great benefit. One of the best example of a mammoth weight gain is with frame-by-frame animations. I’ve been struggling with Android Wear development recently and had a look at the Android wearable support library. Just like the regular Android support library, the wearable variant contains several utility classes when dealing with wearable devices.
Unfortunately, after building a very basic “Hello World” example, I noticed the resulting APK was more than 1.5MB. After a quick investigation into wearable-support.aar, I discovered the library bundles 2 frame-by-frame animations in 3 different densities: a “success” animation (31 frames) and an “open on phone” animation (54 frames).
The frame-by-frame success animation is built with a simple AnimationDrawable defined in an XML file:
The good point is (I’m being sarcastic of course) that each frame is displayed for a duration of 33ms making the animation run at 30fps. Having a frame every 16ms would have ended up with a library twice larger… It gets really funny when you continue digging in the code. The generic_confirmation_00175 frame (line 15) is displayed for a duration of 333ms. generic_confirmation_00185 follows it. This is a great optimization that saves 9 similar frames (176 to 184 included) from being bundled into application. Unfortunately, I was totally disappointed to see that wearable-support.aar actually contains all of these 9 completely unused and useless frames in 3 densities.3
Doing this animation in code obviously requires development time. However, it may dramatically reduce the amount of assets in your APK while maintaining a smooth animation running at 60fps.. At the time of the writing, Android doesn’t provide a easy tool to render such animations. But I really hope Google is working on a new light-weight real-time rendering system to animate all of these tiny details that material design is so fond of. An “Adobe After Effect to VectorDrawable” designer tool or equivalent would help a lot.
Going even further ?
All of the techniques described above mainly target the app/library developers side. Could we go further if we had total control over the distribution chain? I guess we could but that would mainly involve some work server-side or more specifically Play Store-side. For instance, we could imagine a Play Store packaging system that bundles only the native libraries required for the target device.
On a similar note, we could imagine only packaging the configuration of the target device. Unfortunately that would completely break one of the most important functionalities of Android: configuration hot-swapping. Indeed, Android has always been designed to deal with live configuration changes (language, orientation, etc.). For instance, removing resources that are not compatible with the target screen density would be a great benefit. Unfortunately, Android apps are able to deal on the fly with a screen density change. Even though we could imagine deprecating this capability, we would still have to deal with drawables defined for a different density than the target density as well as those having more than a single density qualifier (orientation, smallest width, etc.).
Server-side APK packaging looks extremely powerful. But is is also very risky because the final APK delivered to the user would be completely different from the one sent to the Play Store. Delivering an APK with some missing resources/assets would just break apps.
Conclusion
Designing is all about getting the best out of a set of constraints. The weight of an APK file is clearly one of these constraints. Don’t be afraid of pulling the strings out of one apsect of your application to make some other better in some ways. For instance, do not hesitate to reduce the quality of the UI rendering if it reduce the size of the APK and make the app smoother. 99% of your users won’t even notice the quality drop while they will notice the app is light-weight and smooth. After all, your application is judged as a whole, not as a sum of severed aspects.
1 The .aar library extension is a pretty similar archive. The only difference being that the files are stored in a regular non-compiled jar/xml form. Resources and Java code are actually compiled at the very moment the Android application using them is built.
2 There is just one optional exception to this rule: the launcher icon. The new Google experience launcher relies on the density “above” the current screen density to render the icon on the launcher. Thus, I always bundle an xxxhdpi version of this icon.
3 I personally consider this as a huge flaw in the Android wearable support library and decided not to use it. I couldn’t afford adding a 1.5MB Android Wear app to my 3.5MB Android app (especially knowing it is sent to devices probably not having a connected Android Wear device). As a solution, I re-implemented on my own the only interesting utilities.
1 package com.test;
2
3 /**
4 * Test Demo
5 */
6 public class HelloWorld {
7
8 public static void main(String[] args) {
9 if (BuildConfig.FLAG) {
10 System.out.println("Hello World");
11 } else {
12 System.out.println("Java条件编译测试");
13 }
14 }
15
16 }
再来看看BuildConfig.java文件:
package com.test;
/**
* Created by Administrator on 2015/5/24.
*/
public final class BuildConfig {
public static final boolean FLAG = false;
}
再来看看该.java文件被转化为.class文件的内容:
//
// Source code recreated from a .class file by IntelliJ IDEA
// (powered by Fernflower decompiler)
//
package com.test;
public class HelloWorld {
public HelloWorld() {
}
public static void main(String[] args) {
System.out.println("Java条件编译测试");
}
}
package com.test;
/**
* Test Demo
*/
public class HelloWorld {
public static void main(String[] args) {
if (BuildConfig.FLAG.equals("false")) {
System.out.println("Hello World");
} else {
System.out.println("Java条件编译测试");
}
}
}
HelloWorld.class文件:
//
// Source code recreated from a .class file by IntelliJ IDEA
// (powered by Fernflower decompiler)
//
package com.test;
public class HelloWorld {
public HelloWorld() {
}
public static void main(String[] args) {
if("false".equals("false")) {
System.out.println("Hello World");
} else {
System.out.println("Java条件编译测试");
}
}
}
如果真的需要字符串比较,实现条件编译的话可以使用 “==”。
HelloWorld.java文件:
package com.test;
/**
* Test Demo
*/
public class HelloWorld {
public static void main(String[] args) {
if (BuildConfig.FLAG == "false") {
System.out.println("Hello World");
} else {
System.out.println("Java条件编译测试");
}
}
}
HelloWorld.class文件:
//
// Source code recreated from a .class file by IntelliJ IDEA
// (powered by Fernflower decompiler)
//
package com.test;
public class HelloWorld {
public HelloWorld() {
}
public static void main(String[] args) {
System.out.println("Hello World");
}
}