Quick Start
Setup
- Clone the project code(https://github.com/bytedance/ByteX) to your computer and checkout a new branch for developing.
- ByteX project is composed by each module,it means that one plugin corresponds to one module.So. just create a new java library module.
- And then,configare in the module's build.gradle
apply from: rootProject.file('gradle/plugin.gradle')
//extra dependencies
dependencies {
compile project(':TransformEngine')
implementation "br.usp.each.saeg:asm-defuse:0.0.5"
}
The plugin module is ready now!!!
Custom Plugin
- Crate a new plugin based on ByteX, you need to create at least two classes
- One is Extension:It is much more like java bean which is used to configure the plugin
- Another is Plugin,It must implement the Plugin interface. Of course you can implement this Plugin in a simpler way:inherit from the abstract class AbsMainProcessPlugin or CommonPlugin directly. A simple example is shown as follow:
public class SourceFileKillerPlugin extends CommonPlugin<SourceFileExtension, SourceFileContext> {
@Override
protected SourceFileContext getContext(Project project, AppExtension android, SourceFileExtension extension) {
return new SourceFileContext(project, android, extension);
}
@Override
public boolean transform(@Nonnull String relativePath, @Nonnull ClassVisitorChain chain) {
//We are going to modify the bytecodes, so we need to register a ClassVisitor
chain.connect(new SourceFileClassVisitor(extension));
return super.transform(relativePath, chain);
}
@Nonnull
@Override
public TransformConfiguration transformConfiguration() {
return new TransformConfiguration() {
@Override
public boolean isIncremental() {
//The plugin is incremental by default.It should return false if incremental is not supported by the plugin
return true;
}
};
}
}
- After creating the plugin, you need to make gradle recognize our plugin. We have two configurate methods shown as follow:
- Use Annotation(recommanded)
@PluginConfig("bytex.sourcefile")
public class SourceFileKillerPlugin extends CommonPlugin {...}
- write properties file
We need to create a properties file in the resource directory. The file name of the properties file corresponds to the id of the plugin. The id of the plugin is decided by yourself but it must be unique in the project. As is shown in the figure below, the properties file name is bytex.sourcefile.properties, you can configure it like this:
You need to configure the full class name (package name + class name) of our Plugin class in the properties file, for example:
Then you need to configure this plugin to use this plugin in your app project:
Now, our new plugin has begun to take shape, to make our plugin process class files, you need to define the corresponding ClassVisitor or directly operate the ClassNode.
Publish Plugin
- publish to local
Execute the publish script in the root directory of the project.
./publish.sh
Or double-click uploadArchives to publish the plugin to the gralde_plugins directory of the local project.
You need publish the pulgin again if you want to take effect the changed code.
- publish to maven
If the plugin has been developed and passed the test, we need to publish the plugin to the maven to integrate it into the actual project. First,create a new local.properties configuration file in the root directory of ByteX and add the following configuration:
UPLOAD_MAVEN_URL=xxx
UPLOAD_MAVEN_URL_SNAPSHOT=xxx
USERNAME=xxx
PASSWORD=xxx
USERNAME_SNAPSHOT=xxx
PASSWORD_SNAPSHOT=xxx
Then, upgrade the upload_version of ext.gradle.
Similarly, execute the script or double-click uploadArchives to publish the plugin to online maven.
./publish.sh -m
- publish to snapshot
version=$current_version-${user.name}--SNAPSHOT
./publish.sh -m -t
Debug Plugins
Create a new 'run configuration' in AndroidStudio.
After publishing the plugin locally and connecting it to the app project, append parameters at the end of the build-command before executing the build-command.For example:
./gradlew clean :example:assembleDouyinCnRelease -Dorg.gradle.debug=true --no-daemon
Then switch to the Configuration that you created just now, and click the debug button.
Demo
SourceFileKiller is a custom plugin with less code and can be used as a demo. It does very simple things:delete SourceFile and line number attributes in bytecodes
For External Project
If you need to develop a plugin based on ByteX in external project , you need to configure dependencies like below:
compileOnly "com.android.tools.build:gradle:$gradle_version"
compile "com.bytedance.android.byteX:common:${bytex_version}"
If you want to register the plugin with annotations, you can introduce the following dependencies (optional):
kapt "com.bytedance.android.byteX:PluginConfigProcessor:${bytex_version}"
Primary API
Modify Class With ASM
ByteX is based on ASM, we can read and write class files by registering ClassVisitor or operating ClassNode directly when processing class files. (If you need to receive the bytecode of a file as input, you can refer to the Advanced API below).
By default, the Transform formed by ByteX has at least one regular process (MainProcess) for Class files which includes the following steps:
- traverse callback:Iterate through all the build products (class files and jar class) in the project once, do analysis only and without modifying the input files;
- traverseAndroidJar callback:Iterate through all the class files in android.jar (The version of android.jar is determined by the target api in the project). It is designed for building a complete class diagram.
- transform callback:Iterate through all the build products in the project again, process the class file and output it (It may be directly written to the file as transform outputs or be used as input for the next process).
So, one process will traverse through all the classes in the project twice. We call the transform process as one TransformFlow. Developers can design their own TransformFlow( it can contain multiple traverses, or only contain transform classes but no traverse, etc.), please refer to the Advanced API.
Let's back to the SourceFileKiller Plugin we talked before, the plugin inheriteds from CommonPlugin,if we need to process the class file during the transform phase (the third step), the Plugin class needs to override one of the following two methods:
* transform all the classes in the project
*
* @param relativePath relativePath of the class
* @param chain object for ClassVisitor registration
* @return if return true, this class will be outputed ;if return false, this class will be deleted.
*/
@Override
public boolean transform(String relativePath, extension chain) {
chain.connect(new SourceFileClassVisitor(context));
return true;
}
/**
* transform all the classes in the project
*
* @param relativePath relativePath of the class
* @param node classNode which contains all class infos.
* @return if return true, this class will be outputed ;if return false, this class will be deleted.
*/
@Override
public boolean transform(String relativePath, ClassNode node) {
// do something with ClassNode
return true;
}
We can see that the only difference between these two overloaded methods is their input parameters.The former uses ASM's ClassVisitor, and the latter uses ASM's Tree API, which can directly handle ClassNode.
Similarly, if we need to analyze the class file during the traverse phase, the Plugin class can override the following methods:
* traverse all the classes in the project
*
* @param relativePath relativePath of the class
* @param chain object for ClassVisitor registration
*/
void traverse(@Nonnull String relativePath, @Nonnull ClassVisitorChain chain);
/**
* traverse all the classes in the project
*
* @param relativePath relativePath of the class
* @param node classNode which contains all class infos.
*/
void traverse(@Nonnull String relativePath, @Nonnull ClassNode node);
Log
We recommend developers record all the modifications the plugin did while transforming the classes into logs. We can find out what has been modified by our plugins and find bugs through these logs. Each plugin has its own logger and log file which provided by ByteX. Developers can record all changes by ByteX logger as easily as common Logger When you need to record logs, you could obtain the Logger object from the context and call the corresponding log method.
The logs will be recorded in the file whose path will locate at the app/build/ByteX/$ {variantName}/${extension_name}/${logFile}. If the logFile is not configured in gradle, the file name will use $(extension_name)_log.txt by default.
At the same time, a visual html log file will be generated as transform ends. The data of this html page comes from plugins of the transform, developers don`t need to care about it, it is generated automatically. The file locates at app/build/ByteX/ByteX_report_{transformName}.html.
Tips: If the plugin has needs to generate extra files, we recommend developers use context.buildDir() to get a directory to place the files, This directory locates at the app/build/ByteX/$ {extension_name}/
Advanced API
TransformFlow
In order to provide more flexibility for ByteX-based plugins, we introduce the concept of TransformFlow.
The process of processing all the build products (usually class files) is defined as a TransformFlow. A plugin can run in an independent TransformFlow , or you can take a ride of the global MainTransformFlow(Traverse, traverseAndroidJar and transform form a MainTransformFlow).
You need to override provideTransformFlow which is a method belongs to IPlugin if you want use a customized TransformFlow for the plugin.
@Override
protected TransformFlow provideTransformFlow(@Nonnull MainTransformFlow mainFlow, @Nonnull TransformContext transformContext) {
return mainFlow.appendHandler(this);
}
// create a MainTransformFlow which is independent of the global MainTransformFlow
@Override
protected TransformFlow provideTransformFlow(@Nonnull MainTransformFlow mainFlow, @Nonnull TransformContext transformContext) {
return new MainTransformFlow(transformer, new BaseContext(project, android, extension));
}
// create a customized TransformFlow which is independent of the global MainTransformFlow
@Override
protected TransformFlow provideTransformFlow(@Nonnull MainTransformFlow mainFlow, @Nonnull TransformContext transformContext) {
return new AbsTransformFlow(transformer, new BaseContext(project, android, extension)) {
@Override
protected AbsTransformFlow beforeTransform(Transformer transformer) {
return this;
}
@Override
protected AbsTransformFlow afterTransform(Transformer transformer) {
return this;
}
@Override
public void run() throws IOException, InterruptedException {
// do something in flow.
}
};
}
Class Diagram
Normally,there should be a Class Diagram in each TransformFlow which constains all class relationship between classes of project, classes of jar and classes of Android.jar,it depends on your implementation of TransformFlow.
When your plugin run in MainTransformFlow (the default is this TransformFlow), the plugin will generate the class diagram of this TransformFlow automatically after traverse (including traverseArtifactOnly and traverseAndroidJarOnly), and the diagram will be placed in the corresponding context object. Class graph object can be obtained by call context.getClassGraph().
protected final Project project;
protected final AppExtension android;
public final E extension;
private ILogger logger;
private Graph classGraph;//class diagram
...
public Graph getClassGraph() {
return classGraph;
}
}
Notes:
- In cases when TransformFlow does not complete the traverse (to be precise, beforeTransform of CommonPlugin), the class diagram does not exist, and the object that obtains the class diagram will be null.
- Every plugin that inherits from CommonPlugin must call the corresponding super method if it overrides the beforeTransform method, otherwise the class diagram object will not be passed to the Context object of the current plugin.
- The two TransformFlow class diagrams are isolated. Generally, each TransformFlow will modify the classes, the class diagrams generated by the two TransformFlows are generally different.
File Locator
ByteX has basic reading and output capabilities for INPUTS.If you need to obtain more information such as transform inputs, project inputs, and input aars, you can use the capabilities provided by the Engine layer to obtain the corresponding file inputs.
You can get all the inputs of the transform by following way:
context.getTransformContext().allFiles()
You can get all the merged resources of the project by following way:
context.getTransformContext().getArtifact(Artifact.MERGED_RES)
You can get the location of the file by scope by following way:
context.getTransformContext().getLocator().findLocation("${className}.class",SearchScope.ORIGIN)
MainProcessHandler
MainProcessHandler is bound to the MainTransformFlow processor, and each class will be processed in each step by calling the corresponding method of MainProcessHandler for processing. Generally, our plugins have already implemented this interface, and developers can override the corresponding methods to get the corresponding callbacks.
-
There are series of methods like init, traverse, and transform all process class files through ASM in the MainProcessHandler. In order to provide greater flexibility, you can register your own FileProcessor by overriding the
Listmethod .process (Process process) -
There a method named flagForClassReader in the MainProcessHandler which could customize the flag passed in when ClassReader calls the accept method to read the class file.The default value is
ClassReader.SKIP_DEBUG
FileProcessor
If developers don`t want to use the upper-layer interface encapsulated by ASM to process class files, ByteX also provides a low-level API-FileProcessor.
FileProcessor is similar to the interceptor design of OkHttp. Each class file will be processed through a series of FileProcessors. The advantages of using this interface is that it is more flexible! As an interceptor, you can make the subsequent FileProcessor finish processing before processing, or you can even process it without passing it to the following FileProcessor.
Output process(Chain chain) throws IOException;
interface Chain {
Input input();
Output proceed(Input input) throws IOException;
}
}
public class CustomFileProcessor implements FileProcessor {
@Override
public Output process(Chain chain) throws IOException {
Input input = chain.input();
FileData fileData = input.getFileData();
// do something with fileData
return chain.proceed(input);
}
}
To register a custom FileProcessor, we also provide a more convenient way:Register the FileProcessor with the annotation @Processor on the custom Plugin.
@Processor(implement = CustomFileProcessor.class, process = Process.TRAVERSE)
public class CustomPlugin extends CommonPlugin<Extension, Context> {...}
FileHandler
FileHandler is an interface that is further encapsulated by FileProcessor.The input parameter is a FileData, and the FileData contains the bytecode of the file.
void handle(FileData fileData);
}
public class CustomFileHandler implements FileHandler {
@Override
public void handle(FileData fileData) {
// do something with fileData
}
}
To register a custom FileHandler, just like FileProcessor, we also provide a more convenient way:Register the FileHandler with the annotation @Handler on the custom Plugin.
@Handler(implement = CustomFileHandler.class, process = Process.TRAVERSE)
public class FlavorCodeOptPlugin extends CommonPlugin<Extension, Context> {...}
TransformConfiguration
The methods in this interface correspond to the methods in the Transform interface in the Transform API.
Each plugin can customize some configurations by overriding the corresponding interface method belongs to transformConfiguration.
For example:
public TransformConfiguration transformConfiguration() {
return new TransformConfiguration() {
@Override
public Set<QualifiedContent.ContentType> getInputTypes() {
return TransformManager.CONTENT_JARS;
}
};
}
hookTransform/Task
Using transform api by calling registerTransform could only run your plugin before proguard or dexBuilder.If you have a plugin which must run after proguard execution,or run before(after) any task execution,you could apply the HookTransform/Task feature provided by ByteX.ByteX provide two methods(all named as hookTask) to complete the feature.(The hookTransformName method design in previous versions is still supported but marked as Deprecated,and ByteX give priority to this solution,the new solution will work on if the plugin hooks a non-transform task or runs after task execution)
For example, if you want your plugin to be executed after proguard (before dexBuilder), you can do this:
/**
* eg:dexBuilder
* use {@link #hookTask()} and {@link #hookTask(Task)} instead
*/
@Deprecated
@Nullable
String hookTransformName() {
return "dexBuilder";
}
/**
* use hook mode.false by default
*/
boolean hookTask() {
return true;
}
/**
* How to hook this task
*
* @param task
* @return {@link HookType#Befor:plugin will run before task execution
* {@link HookType#After} :plugin will run after task execution
* {@link HookType#None} do not hook the task
*/
@Nonnull
HookType hookTask(@Nonnull Task task) {
if(task.getName().contains("dexBuilder")){
return HookType.Before;
}
return HookType.None;
}
}
Obfuscate Tool
Some plugins may run after proguard execution,and these plugins usually need doing something like obfuscated or de-obfuscated.ByteX provides a set of class-tools for parsing proguard-mapping:
//obtain mapping file
File mappingFile = context.getTransformContext().getProguardMappingFile();
//read and parse mapping file
MappingReader mappingReader = MappingReader(mappingFile);
MappingProcessor mappingProcessor = new FullMappingProcessor();
//The class name in the mapping file is separated by '.', but internal name or desc is commonly used in asm, so we provide a MappingProcessor adapter
mappingProcessor = new InternalNameMappingProcessor(mappingProcessor);
mappingReader.pump(mappingProcessor);
//At the same time, we provide a simple retrace for asm users
FullInternalNameRetrace retrace = new FullInternalNameRetrace(mappingFile);