Learning Atlas for C

From AtlasWiki
Revision as of 15:37, 22 June 2017 by Admin (Talk | contribs) (Call Graph in the Smart View)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Software Engineers have long used Integrated Development Environments (IDEs) to automate tasks in the edit-compile-debug cycle. As software becomes larger, merely navigating the web of code becomes a serious obstacle to understanding it.

Atlas is an unprecedented leap in IDE technology to expedite ad hoc searches and to scale up systematic audits. It provides a unique capability to analyze and visualize software by representing the program as a unified graph of nodes and edges. With an Atlas Smart View, using an analysis is as simple as clicking on the code. For example, clicking a function name displays a call graph. Program analysis experts may define and seamlessly integrate new analyses into the unified graph, delivering their solutions to fellow developers. For in-between tasks, the Atlas Shell can be used to write sophisticated queries.

This tutorial will get you started with Atlas for C. It was last tested with: Atlas 3.1.3.

Demo Project Setup

This tutorial is based on preconfigured demo projects. To import one of the available demo projects, navigate to: File > New > Other > Import Demo Project.

LearningAtlasForC 0.png

A wizard with a list of available demo projects will be displayed. This tutorial series is based on the XINU demo project which is a small operating system written in C. Once imported into your workspace, Atlas will automatically map the project. You may notice a small progress bar at the bottom of Eclipse indicating the mapping percentage. Once complete, your demo project is mapped and ready for use with Atlas.

Later on, you may refer to Configuring C Projects to set up your own code to work with Atlas.

Atlas Smart View

Need to understand code someone else wrote? Or plan a major change? The Atlas Smart View can quickly shed light on how your code works and how different parts are interconnected. You can click anywhere in your code and instantly get visualization of your code relevant to the artifact you clicked on. The Smart View can quickly answer questions like:

  • If I change this method, how will other methods be affected?
  • Where does the value of this parameter come from?

This video shows you how easy it is to get started with the Atlas Smart View. In addition, the following walkthrough will introduce you to Smart Views using XINU.

Open a Smart View

To open an Atlas Smart View, navigate to: Atlas > Open Smart View. You may open multiple Atlas Smart Views at the same time, and configure each view separately. Atlas Smart Views come with built-in scripts that are ready to use. To select a script, pick one from the combo box at the bottom of the Smart View.

Call Graph script in the Smart View

The Call script in the Smart View shows call relationships between functions, otherwise known as a call graph. To use it, first select Call from the combo box located at the bottom of the Atlas Smart View pane.

To demonstrate the common scripts, we will use Xinu/sys/dswrite.c. To continue, open it by double clicking on the file in the Project Explorer. Then click on the function name dswrite in the source code editor.

The Smart View responds to selections by showing the call graph immediately around the dswrite function. The node corresponding to the dswrite function is highlighted in yellow; the enclosing nodes represent the translation unit and project. From the call graph, we see that dswrite calls dskenq and getbuf. The edge label 1 indicates that there is a single call site in dswrite for the called function.

LearningAtlasForC 1.png

The call graph can be used to navigate back to the source code. Double-clicking a function node leads to the function definition. Double-clicking on a call edge leads to the call sites where the successor was called in the predecessor.

Smart View Controls

Continuing using the call graph, note the gray dashed edge from dskenq - this indicates that dskenq calls other functions. The control on the left edge of the Smart View can be used to expand the call graph in the forward and reverse directions, showing more callers and callees.

The top expansion control increases/decreases the number of callers. In other words, it walks in the reverse direction of the call edges. The same applies to the bottom expansion control, which increases/decreases callees. By expanding two times in the forward direction, we get the following graph:

LearningAtlasForC 2.png

You can reset the expansion by clicking on the diamond button ♦ in the middle of the expansion controls. You can also expand an individual gray edge by double-clicking on it.

The Atlas Smart View enables you to change the layout of the displayed graph to either: hierarchical, or orthogonal.

LearningAtlasForC 3.png

You can also fit the graph within the Atlas Smart View pane, zoom-in, zoom-out, or zoom to the selection (e.g. dswrite in our example).

LearningAtlasForC 4.png

When you obtain a particularly useful graph, you can make a copy using the pin button. The copy can then be used to make further selections for Smart Views.

LearningAtlasForC 5.png

Atlas provides you with means to export the current graph to an image. The available image extensions are: .jpg, .png, and .svg.

LearningAtlasForC 6.png

Control Flow script in the Smart View

The Control Flow script in the Smart View displays control flow graphs. To get started, select the Control Flow script from the combo box at the bottom of the Smart View. Statements are represented by control flow nodes, nested inside functions. Selecting the statement drptr->drdba = block; at line 23 inside the dswrite function will show the following control flow graph:

LearningAtlasForC 7.png

You can see the flow of statements from and to the selected statement which is highlighted in yellow. The expansion controls work as before, but here the traversal is using control flow edges instead of call edges as in the Call script in the Smart View.

Data Flow script in the Smart View

The Data Flow script in the Smart View displays data dependencies between variables and expressions. This includes intra-procedural data flow (within a function) and inter-procedural data flow (across functions). To get started, select the Data Flow script from the combo box.

Next, select a variable or expression. For example, we are interested in tracking the global variable dskrbp which is passed to getbuf in the dswrite function. We can simply select the dskrpb variable in the editor and the corresponding data flow graph will be displayed:

LearningAtlasForC 8.png

From the graph, we can see that dskrbp is not a local variable because it is not contained within the boundaries of the dswrite function, and that the variable is passed as a parameter to the function getbuf in the statement drpts = (struct dreq *) getbuf(dskrbp). It also shows that the parameter it is passed to is named poolid. Expanding forward on the data flow graph, we can see how the variable is propagated and manipulated through the different statements in the getbuf function.

Beyond Smart Views

Atlas offers additional built-in Smart Views, and an extension mechanism for creating your own Smart Views. While the built-in Smart Views may be enough for your purposes, and creating new Smart Views may be too much, there is also an Atlas Shell for in-between tasks.

Atlas Shell

The Atlas Shell provides a way to invoke Atlas queries and display results, without the overhead of creating a Smart View or an Atlas Plug-in.

The Atlas Shell (shown below) uses a Scala interpreter preconfigured for use with Atlas. Scala is a JVM language that integrates seamlessly with Java, the the language used to implement Eclipse and Atlas. The easiest way to learn how to use the Atlas Shell to write queries is by example, so let’s get started.

LearningAtlasForC 9.png

Opening an Atlas Shell

To open the Atlas Shell, navigate to Atlas > Open Atlas Shell.

Your First Query

In Atlas, the entire software graph is referred to as the universe. For your first Atlas query, ask for everything. Go ahead, it won't take long. We promise. Type this in the Evaluate box and press enter:

universe

In response, you'll see something like:

res0: com.ensoftcorp.atlas.core.query.Q = <Atlas query expression>

What happened?

First, we did not specify what to assign the result to, so the Atlas Shell helpfully made a new variable called res0 for the result. The type of the result is com.ensoftcorp.atlas.core.query.Q, and the string representation is <Atlas query expression>.

Evaluating the Universe

The expression universe is a starting point for Atlas queries based on Q. In short, Q is a way to build up expressions which specify what you want, but nothing happens until you evaluate and start enumerating the result.

Evaluating Q will yield a Graph, which has a set of nodes and a set of edges. Try that next.

universe.eval()

The result is a Graph.

res1: com.ensoftcorp.atlas.core.db.graph.Graph = ....

Let's find out how many nodes and edges are in the universe, shall we? Assign the Graph to g. In Scala, a variable is declared with the keyword var, and the type is inferred.

var g = universe.eval();
g.nodes().size();
g.edges().size();

You should see counts for the number of nodes and edges, respectively. The key point here is that we write queries using Q, and eval() will return a Graph with nodes and edges.

Seek and Display

Let's try some more interesting queries. Recall that evaluating a Q results in a Graph. Graphs have nodes and edges, which are represented by Node and Edge classes. Nodes and edges have both attributes and tags. The values of attributes and tags can be specified using queries. The schema for the graph is the eXtensible Common Software Graph (XCSG).

For the next example, let’s get the node for the dswrite function. To do that in Atlas in terms of XCSG, we need to search for a node having an attribute name equal to dswrite, and tagged Function to avoid matching other artifacts which incidentally share the name.

First, we select all the nodes that are functions, or in Atlas terminology, the nodes tagged with XCSG.Function.

// select by tag
var functions = universe.nodes(XCSG.Function)

Second, of these function nodes, select all the nodes with a given name using the query for node attributes, Q.selectNode().

// select by name
var dswrite = functions.selectNode(XCSG.name, "dswrite")

Finally, we show the result:

show(dswrite)

LearningAtlasForC 10.png

You will see the function dswrite, along with its parent package and project. This is because show() is automatically providing some visual context by including the nodes which point to the result via an XCSG.Contains edge.

If you want to see the exact result of your query, which is sometimes helpful for debugging, you can tell show() not to extend your result.

show(function, false)

Call Graph Using Atlas Shell

Getting the call graph is a traversal over edges. To get the call graph like the one produced by the Call script in the Smart View with only one-level on the forward and backward directions, you can simply invoke the following lines in Atlas Shell:

var dswrite = universe.nodes(XCSG.Function).selectNode(XCSG.name, "dswrite");
var cg = edges(XCSG.Call).forwardStep(dswrite);
var rcg = edges(XCSG.Call).reverseStep(dswrite);
var callGraph = cg.union(rcg);
show(callGraph);

The first line finds the function node corresponding to dswrite, filling in the role of selecting the function from the source code editor, but here we directly select the function node from the Atlas graph. The second line walks a step forward on the Call edges from the dswrite function node. The third line walks a step in the reverse direction. The fourth line unions both results to form the single-step forward/backward call graph. The last line displays the corresponding graph in an Atlas Graph View.

LearningAtlasForC 11.png

Adding Some Color

A traversal might result in a large graph, so it may be helpful to add some color to call attention to where the traversal started. You can do that with a Markup. A Markup associates a color with a query expression, which is then passed to show() via the Markup parameter.

For example, to add a color to the dswrite function node in the previous call graph, we can invoke the following queries:

// highlight the origin in red
var markup = new Markup();
markup.setNode(dswrite, MarkupProperty.NODE_BACKGROUND_COLOR, java.awt.Color.RED);

// display
show(callGraph, markup);

Note that you do not need to re-invoke the previous script to construct the call graph again - the callGraph variable holds the call graph we constructed before, so we only need to add coloring to it.

LearningAtlasForC 12.png

Data Flow Using Atlas Shell

Suppose that you want mimic the behavior of the Data Flow Smart View to find out where the values of a field within a specific structure flows to. Atlas provides data flow edges which you can query to find out. For this example, we are interested in tracking drbuff fields in structure dreq. To query the drbuff field, we will use the convenience method Common.fieldSelect(). We can always go an alternative route by selecting the nodes tagged with XCSG.Field and named drbuff and contained (XCSG.Contains) within the structure dreq.

Starting from the field drbuff, we can walk forward over all the data flow edges (tagged XCSG.DataFlow_Edge). For simplicity, we will walk 3 steps forward on the data flow edges from drbuff.

var src = fieldSelect("struct dreq", "drbuff");
var dataFlowGraph = src.forwardStepOn(edges(XCSG.DataFlow_Edge),3);
show(dataFlowGraph);

Note the use of the alternate traversal query forwardStepOn, which starts from src and walks within the given edges, which is the reverse syntax from forwardStep.

LearningAtlasForC 13.png

In the resulting graph, the data flow nodes are colored cyan, and appear nested in control flow blocks, which are colored in light blue. For the example code, the forward step data flow from drbuff field will show that the field has been aliased multiple times and passed to a function freebuf one time. The tag XCSG.DataFlow_Edge incorporates two kinds of data flow edges: XCSG.LocalDataFlow and XCSG.InterproceduralDataFlw, which are displayed with edge labels df(local) and df(inter), respectively. Local data flow represents flows within a method, while interprocedural represents flows between methods, including flows to parameters, from the return statement of a method, or to and from fields.

Atlas Plug-ins

While the Atlas Shell is convenient for one-off queries, eventually you may want to write longer programs based on Atlas queries, or contribute your own Smart Views. To do that, you will create an Atlas plug-in.

An Atlas plug-in is just a standard Eclipse plug-in which extends Atlas. Once you have created and installed your plug-in, you can invoke code contributed by your plug-in from the Atlas Shell, or select your custom Smart View.

To get started, Atlas provides a wizard to create a new sample Atlas plug-in. To import it, navigate to File > New > Other > Atlas > Sample Atlas Plug-in Project.

LearningAtlasForC 14.png

Give a name for the project, and click Finish.

You will now have an Eclipse plug-in in your workspace. To use it, you will need to launch another instance of Eclipse, which will have it’s own workspace.

Run the Sample Atlas Plug-in in a new instance of Eclipse as follows:

1. Open the Run Configurations dialogue from the context menu on the plug-in project containing the custom scripts. Right-click, Run As > Run Configurations ...

LearningAtlasForC 15.png

2. In the Run Configurations dialogue, set the Location to the workspace containing the project you want to analyze. To keep working with Xinu, simply take the default and create a new workspace. After it starts, add the Xinu project to the new workspace.

LearningAtlasForC 16.png

3. Click Run. A new instance of Eclipse will be launched.

Custom Atlas Programs

In the first workspace, in the sample plug-in under the source package, you will find the sample Atlas program CallGraphAtlasScript.java. The class contains a static method create(). When invoked, create() iterates through each function in the mapped project(s), creates call graph of immediate callers and callees for the function, and saves the graph to the "call-graphs" folder in the user’s home directory.

Because this code is an Eclipse plug-in, we have many options for invoking it, such as creating a menu item. For this example, we will explain how to invoke it from the Atlas Shell.

In the new Eclipse instance, open an Atlas Shell, and click on the Settings button in the toolbar. In Shell Dependencies, add SampleAtlasPluginProject to the list of Selected plug-ins. In Initialization Script, add the line import com.ensoftcorp.demo.CallGraphAtlasScript. Click OK. The Atlas Shell now has the sample plug-in on the classpath, and automatically imports the CallGraphAtlasScript so we can use the unqualified name.

Finally, to run the program, type CallGraphAtlasScript.create(). After it completes, the folder "call-graphs" in your home directory will have images of the call graphs.

Contributing Custom Smart Views

Custom Smart Views can be contributed to Atlas via an Eclipse OSGI extension. To do so, you need to have a Java plug-in project. The Sample Atlas Plug-in project contains three custom Smart Views for you to learn from. If you have not already created the sample project and started a new Eclipse instance, refer to the instructions at the start of this section.

The Sample Atlas Plug-in Project comes with three different custom Atlas Smart Views: Data Flow (within a function), TypeOf, and Custom Call Graph. Navigate to Atlas > Open Smart View, and you will see your custom Smart View scripts are added to the combo box. The new Smart Views are described below.

Example: Data Flow (within a function)

The Data Flow (within a fuction) (DataFlowWithinFunctionSmartView.java) script listens to selections of functions, then it displays the data flow graph embedded in the control flow graph. For example, selecting the Data Flow (within a function) script from the drop-down menu box and clicking on dswrite function name will display the following graph:

LearningAtlasForC 17.png

Example: TypeOf

The TypeOf (TypeOfSmartView.java) script listens to selections of types or a variables, then displays the immediate type and the basis of that type if available. For example, selecting the TypeOf script, and clicking on the drptr variable will show the type hierarchy of the variable. It shows that the variable is of pointer type * from structure dreq.

LearningAtlasForC 18.png

Example: Custom Call Graph

The Custom Call Graph (CallGraphSmartView.java) script listens to selections of structures, then it constructs the induced call graph between the functions which reference the structure. For example, suppose that we have an Atlas Graph displayed containing the node for structure dreq, which can be obtained using the following Atlas Shell script:

var structs = universe.nodes(XCSG.C.Struct)
var dreqStruct = structs. selectNode(XCSG.name, "struct dreq")
show(dreqStruct)

Then, we select the dreq node from the displayed Atlas Graph, and the Custom Call Graph will display the call graph between the functions accessing instances of the dreq structure, as shown:

LearningAtlasForC 19.png