Using Automata

Scapy enables to easily create network automata. Scapy does not stick to a specific model like Moore or Mealy automata. It provides a flexible way for you to choose your way to go.

An automaton in Scapy is deterministic. It has states, transitions and actions. There is a start state and some end and error states. Transitions go from one state to another. Transitions can be transitions on a specific condition, transitions on the reception of a specific packet, transitions on an event on a file descriptor or transitions on a timeout. When a transition is taken, one or more actions can be run. An action can be bound to many transitions. Parameters can be passed from states to transitions and from transitions to states and actions.

From a programmer's point of view, states, transitions and actions are methods from an Automaton subclass. They are decorated to provide meta-information needed in order for the automaton to work.

First example

Let's begin with a simple example. I take the convention to write states with capitals, but anything valid with python syntax would work as well.

class HelloWorld(Automaton):
    @ATMT.state(initial=1)
    def BEGIN(self):
        print "State=BEGIN"
    
    @ATMT.condition(BEGIN)
    def wait_for_nothing(self):
        print "Wait for nothing..."
        raise self.END()
    
    @ATMT.action(wait_for_nothing)
    def on_nothing(self):
        print "Action on 'nothing' condition"
    
    @ATMT.state(final=1)
    def END(self):
        print "State=END"

In this example, we can see 3 decorators:

• ATMT.state that is used to indicate that a method is a state, and that can have initial, final and error optional arguments set to non-zero for special states.
• ATMT.condition that indicate a method to be run when the automaton state reaches the indicated state. The argument is the name of the method representing that state
• ATMT.action binds a method to a transition and is run when the transition is taken.

Running this example gives the following result:

>>> a=HelloWorld()
>>> a.run()
State=BEGIN
Wait for nothing...
Action on 'nothing' condition
State=END

This simple automaton can be described with the following graph:

and the graph can be automatically drawn from the code with

>>> HelloWorld.graph()

Why using automata

What can be done with automata can also be done without them.

Automata give you a recipe and a formalism to cut a big problem into simple ones. As such, you had better draw the automaton's graph before your first line of code.

If you decide to go with the automaton formalism, Scapy's automaton class will exempt you from writing the parts common to all automata and focus on states, transitions and actions.

Scapy engine brings you

• a flexible automaton engine with a consice Python syntax to describe states, transitions and actions.
• reception loops, timeouts, storing of exchaged packet, etc. already included.
• integrated debugging logs. No need to add a print "state=XXX" at the begining of each state, just run the automaton with debug=x with 1<=x<=5 as a parameter.
• ability to control the automaton run, put breakpoints on states, intercept and modify sent packets on the fly. Thus you can write a working automaton and have it fuzz any given state of a protocol without modifying it.

Controlling an automaton

An automaton is run as a background thread. When the automaton is instantiated, the background thread is created and awaiting orders from the foreground. Orders are transmited by calling automaton's methods.

An automaton can have 3 states

• running: the automaton is processing inputs, can change its states, etc.
• freezed: the automaton is freezed in the state it was in when interrupted, and does process anything. It is only waiting for an order to unfreeze.
• stopped: there is not even a background thread.

When instantiated, the automaton instance is automatically started and freezed ready to run the initial state.

The controlling methods are

• run() has the automaton run in background and wait for it to finish. Ctrl-C will stop the wait and freeze the automaton.
• runbg() has the automaton run in background but do not wait anything
• next() have the automaton run in background and wait for the next state to be reached. Then the automaton is freezed.
• stop() destroy the background thread
• start() create the background thread
• restart() recreate the background thread

run() and next() can be interrupted by the following exceptions:

• Automaton.ErrorState when an error state is reached
• Automaton.Stuck when the automaton is stuck in a dead-end state
• Automaton.Breakpoint when a breakpointed state is reached
• Automaton.Singlestep when, while singlesteping the automaton (with next()), a new state is reached
• Automaton.InterceptionPoint when a packet is sent while the automaton is in an intercepted state

Singlestepping

Singlestepping is done using the next() method instead of the run() method.

Let's use this new automaton

class Example(Automaton):
    @ATMT.state(initial=1)
    def STATE_1(self):
        raise self.STATE_2()
    @ATMT.state()
    def STATE_2(self):
        self.send(IP(dst="www.example.com")/ICMP())
        raise self.STATE_3()
    @ATMT.state(final=1)
    def STATE_3(self):
        return "job's done"

Singlestepping it will look like this:

>>> a=Example()
>>> a.next()
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "scapy/automaton.py", line 652, in next
    return self.run(resume = Message(type=_ATMT_Command.NEXT))
  File "scapy/automaton.py", line 642, in run
    raise self.Singlestep("singlestep state=[%s]"%c.state.state, state=c.state.state)
Singlestep: singlestep state=[STATE_2]
>>> a.next()
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "scapy/automaton.py", line 652, in next
    return self.run(resume = Message(type=_ATMT_Command.NEXT))
  File "scapy/automaton.py", line 642, in run
    raise self.Singlestep("singlestep state=[%s]"%c.state.state, state=c.state.state)
Singlestep: singlestep state=[STATE_3]
>>> a.next()
"job's done"

Breakpoints

States can be breakpointed or intercepted using instance methods' methods:

>>> a=Example()
>>> a.STATE_2.breaks()
>>> a.run()
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "scapy/automaton.py", line 644, in run
    raise self.Breakpoint("breakpoint triggered on state [%s]"%c.state.state, state=c.state.state)
Breakpoint: breakpoint triggered on state [STATE_2]
>>> a.run()
"job's done"

Breakpoints survive to an automaton restart:

>>> a.restart()
>>> a.run()
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "scapy/automaton.py", line 644, in run
    raise self.Breakpoint("breakpoint triggered on state [%s]"%c.state.state, state=c.state.state)
Breakpoint: breakpoint triggered on state [STATE_2]
>>> a.run()
"job's done"
>>> a.STATE_2.unbreaks()
>>> a.restart()
>>> a.run()
"job's done"

Packet interception

When a state is marked as intercepted, all packets sent while in this state will be submitted for approval. They can then be rejected (nothing will be sent and the automaton will carry on without noticing it), they can be accepted, or they can even be modified (fuzzed, for instance).

Methods are

• accept_packet() to accept (no argument) or modify (the new packet as argument) a packet.
• reject_packet() to reject a packet

By default, these methods will not wait the automaton (like runbg() so that you'll have to run run() after that. But you can provide the wait=True parameter if you want them to wait.

The intercepted packet is available in the packet attribute of the Automaton.InterecptionPoint exception. This will lead to code of this kind

while True:
    try:
        x = a.run()
    except Automaton.InterceptionPoint,p:
        a.accept_packet(Raw(p.packet.load.lower()))
    else:
        break

Because it is not very handy to get that in command line mode, the packet is also available in the automaon's intercepted_packet attribute while the background thread is waiting for the verdict.

In the following example, we will use debugging output to have an idea to what is sent or not.

>>> log_interactive.setLevel(1)
>>> a=Example(debug=5)
DEBUG: Starting control thread [tid=-1232159856]
>>> a.STATE_2.intercepts()

First, let's accept the packet.

>>> a.run()
DEBUG: Received command RUN
DEBUG: ## state=[STATE_1]
DEBUG: switching from [STATE_1] to [STATE_2]
DEBUG: ## state=[STATE_2]
DEBUG: INTERCEPT: packet intercepted: IP / ICMP 10.0.0.1 > Net('www.example.com') echo-request 0
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "scapy/automaton.py", line 645, in run
    raise self.InterceptionPoint("packet intercepted", state=c.state.state, packet=c.pkt)
InterceptionPoint: packet intercepted
>>> a.accept_packet()
DEBUG: INTERCEPT: packet accepted
DEBUG: SENT : IP / ICMP 10.0.0.1 > Net('www.example.com') echo-request 0
DEBUG: switching from [STATE_2] to [STATE_3]
DEBUG: ## state=[STATE_3]
DEBUG: Stopping control thread (tid=-1232159856)
>>> a.run()
"job's done"
>>> 

We can see in the debugging output (DEBUG: SENT: IP / ICMP...) that the packet is sent unmodified. Now let's try to reject it.

>>> a.restart()
DEBUG: Starting control thread [tid=-1232159856]
>>> a.run()
DEBUG: Received command RUN
DEBUG: ## state=[STATE_1]
DEBUG: switching from [STATE_1] to [STATE_2]
DEBUG: ## state=[STATE_2]
DEBUG: INTERCEPT: packet intercepted: IP / ICMP 10.0.0.1 > Net('www.example.com') echo-request 0
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "scapy/automaton.py", line 645, in run
    raise self.InterceptionPoint("packet intercepted", state=c.state.state, packet=c.pkt)
InterceptionPoint: packet intercepted
>>> a.reject_packet()
DEBUG: INTERCEPT: packet rejected
DEBUG: switching from [STATE_2] to [STATE_3]
DEBUG: ## state=[STATE_3]
DEBUG: Stopping control thread (tid=-1232159856)
>>> a.run()
"job's done"

Now let's try to modify it.

>>> a.restart()
DEBUG: Starting control thread [tid=-1232159856]
>>> a.run()
DEBUG: Received command RUN
DEBUG: ## state=[STATE_1]
DEBUG: switching from [STATE_1] to [STATE_2]
DEBUG: ## state=[STATE_2]
DEBUG: INTERCEPT: packet intercepted: IP / ICMP 10.0.0.1 > Net('www.example.com') echo-request 0
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "scapy/automaton.py", line 645, in run
    raise self.InterceptionPoint("packet intercepted", state=c.state.state, packet=c.pkt)
InterceptionPoint: packet intercepted
>>> a.intercepted_packet
<IP  frag=0 proto=icmp dst=Net('www.example.com') |<ICMP  |>>
>>> a.accept_packet(a.intercepted_packet/"XXXXX MODIFIED XXXXX")
DEBUG: INTERCEPT: packet replaced by: IP / ICMP 10.0.0.1 > Net('www.example.com') echo-request 0 / Raw
DEBUG: SENT : IP / ICMP 10.0.0.1 > Net('www.example.com') echo-request 0 / Raw
DEBUG: switching from [STATE_2] to [STATE_3]
DEBUG: ## state=[STATE_3]
DEBUG: Stopping control thread (tid=-1232159856)
>>> a.run()
"job's done"

We can see a Raw layer added to the end of the sent packet.

Writing an automaton

Triggering a state change

The ATMT.state decorator transforms a method into a function that returns an exception. If you raise that exception, the automaton state will be changed. If the change occurs in a transition, actions bound to this transition will be called. The parameters given to the function replacing the method will be kept and finally delivered to the method. The exception has a method action_parameters that can be called before it is raised so that it will store parameters to be delivered to all actions bound to the current transition.

As an example, let's consider the following state:

    @ATMT.state()
    def MY_STATE(self, param1, param2):
        print "state=MY_STATE. param1=%r param2=%r" % (param1, param2)

This state will be reached with the following code

    @ATMT.receive_condition(ANOTHER_STATE)
    def received_ICMP(self, pkt):
        if ICMP in pkt:
            raise self.MY_STATE("got icmp", pkt[ICMP].type)

Let's suppose we want to bind an action to this transition, that will also need some parameters:

    @ATMT.action(received_ICMP)
    def on_ICMP(self, icmp_type, icmp_code):
        self.retaliate(icmp_type, icmp_code)

The condition should become

    @ATMT.receive_condition(ANOTHER_STATE)
    def received_ICMP(self, pkt):
        if ICMP in pkt:
            raise self.MY_STATE("got icmp", pkt[ICMP].type).action_parameters(pkt[ICMP].type, pkt[ICMP].code)

Real example

Here is a real example take from Scapy. It implements a TFTP client that can issue read requests.

class TFTP_read(Automaton):
    def parse_args(self, filename, server, sport = None, port=69, **kargs):

        Automaton.parse_args(self, **kargs)
        self.filename = filename
        self.server = server
        self.port = port
        self.sport = sport

    def master_filter(self, pkt):
        return ( IP in pkt and pkt[IP].src == self.server and UDP in pkt
                 and pkt[UDP].dport == self.my_tid
                 and (self.server_tid is None or pkt[UDP].sport == self.server_tid) )
        
    # BEGIN
    @ATMT.state(initial=1)
    def BEGIN(self):
        self.blocksize=512
        self.my_tid = self.sport or RandShort()._fix()
        bind_bottom_up(UDP, TFTP, dport=self.my_tid)
        self.server_tid = None
        self.res = ""

        self.l3 = IP(dst=self.server)/UDP(sport=self.my_tid, dport=self.port)/TFTP()
        self.last_packet = self.l3/TFTP_RRQ(filename=self.filename, mode="octet")
        self.send(self.last_packet)
        self.awaiting=1
        
        raise self.WAITING()
        
    # WAITING
    @ATMT.state()
    def WAITING(self):
        pass

    @ATMT.receive_condition(WAITING)
    def receive_data(self, pkt):
        if TFTP_DATA in pkt and pkt[TFTP_DATA].block == self.awaiting:
            if self.server_tid is None:
                self.server_tid = pkt[UDP].sport
                self.l3[UDP].dport = self.server_tid
            raise self.RECEIVING(pkt)
    @ATMT.action(receive_data)
    def send_ack(self):
        self.last_packet = self.l3 / TFTP_ACK(block = self.awaiting)
        self.send(self.last_packet)

    @ATMT.receive_condition(WAITING, prio=1)
    def receive_error(self, pkt):
        if TFTP_ERROR in pkt:
            raise self.ERROR(pkt)

    @ATMT.timeout(WAITING, 3)
    def timeout_waiting(self):
        raise self.WAITING()
    @ATMT.action(timeout_waiting)
    def retransmit_last_packet(self):
        self.send(self.last_packet)

    # RECEIVED
    @ATMT.state()
    def RECEIVING(self, pkt):
        recvd = pkt[Raw].load
        self.res += recvd
        self.awaiting += 1
        if len(recvd) == self.blocksize:
            raise self.WAITING()
        raise self.END()

    # ERROR
    @ATMT.state(error=1)
    def ERROR(self,pkt):
        split_bottom_up(UDP, TFTP, dport=self.my_tid)
        return pkt[TFTP_ERROR].summary()
    
    #END
    @ATMT.state(final=1)
    def END(self):
        split_bottom_up(UDP, TFTP, dport=self.my_tid)
        return self.res

It can be run like this, for instance:

>>> TFTP_read("my_file", "192.168.1.128").run()

Detailed documentation

Decorators

Decorator for states

States are methods decorated by the result of the ATMT.state function. It can take 3 optional parameters, initial, final and error, that, when set to True, indicate that the state is an initial, final or error state.

class Example(Automaton):
    @ATMT.state(initial=1)
    def BEGIN(self):
        pass

    @ATMT.state()
    def SOME_STATE(self)
        pass

    @ATMT.state(final=1)
    def END(self):
        return "Result of the automaton: 42"

    @ATMT.state(error=1)
    def ERROR(self):

        return "Partial result, or explanation"
[...]

Decorators for transitions

Transitions are methods decorated by the result of one of ATMT.condition, ATMT.receive_condition, ATMT.ioevent or ATMT.timeout. They all take as argument the state method they are related to. ATMT.timeout also have a mandatory timeout parameter to provide the timeout value in seconds. ATMT.condition, ATMT.receive_condition and ATMT.ioevent have an optional prio parameter so that the order in which conditions are evaluated can be forced. Default priority is 0. Transitions with the same priority level are called in an undetermined order.

When the automaton switches to a given state, the state's method is executed. Then transitions methods are called at specific moments until one triggers a new state (something like raise self.MY_NEW_STATE()). First, right after the state's method returns, the ATMT.condition decorated methods are run by growing prio. Then each time a packet is received and accepted by the master filter all ATMT.receive_condition decorated methods are called by growing prio. Each time a file descriptor watched by a ioevent transition has data, transitions watching it will be called by growing {{{prio}}. When a timeout is reached since the time we entered into the current space, the corresponding ATMT.timeout decorated method is called.

class Example(Automaton):
    @ATMT.state()
    def WAITING(self):
        pass

    @ATMT.condition(WAITING):
    def it_is_raining(self):
        if not self.have_umbrella:
            raise self.ERROR_WET()

    @ATMT.receive_condition(WAITING, prio=1)
    def it_is_ICMP(self, pkt):
        if ICMP in pkt:
            raise self.RECEIVED_ICMP(pkt)
    @ATMT.receive_condition(WAITING, prio=2)
    def it_is_IP(self, pkt):
        if IP in pkt:
            raise self.RECEIVED_IP(pkt)
    
    @ATMT.timeout(WAITING, 10.0)
    def waiting_timeout(self):
        raise self.ERROR_TIMEOUT()

I/O events

I/O events transitions deserver a paragraph of their own.

Let's consider this automaton

class Example(Automaton):
    @ATMT.state(initial=1)
    def BEGIN(self):
        pass
    @ATMT.ioevent(BEGIN, name="my_events")
    def my_transition(self, fd):
        obj = fd.recv()
        print "Got [%r]" % obj
        if  obj == "go":
            raise self.END()
    @ATMT.state(final=1)
    def END(self):
        return "job's done"

We create an I/O event transition that will wait on a file descriptor named my_events. This will automatically create a .io.my_events attribute that we will be able to read and write. From within the automaton, we will be able to manipulate the other ends with the .oi.my_events attributes. The communication channel used is a queue of objects.

We will activate debugging output to follow what is happening.

>>> log_interactive.setLevel(1)
>>> a=Example(debug=2)
>>> a.runbg()
DEBUG: ## state=[BEGIN]

Ok, now we reached the BEGIN state. Let's look at the communication channel:

>>> a.io.my_events
<scapy.automaton._IO_mixer instance at 0xa9db44c>
>>> a.io.my_events.send("hello")
Got ['hello']
DEBUG: I/O event [my_transition] not taken
>>> a.io.my_events.send({1:2,3:4})
Got [{1: 2, 3: 4}]
DEBUG: I/O event [my_transition] not taken
>>> a.io.my_events.send("go")
Got ['go']
DEBUG: I/O event [my_transition] taken to state [END]
DEBUG: switching from [BEGIN] to [END]
DEBUG: ## state=[END]
>>> a.run()
"job's done"

When needed, the named I/O event file descriptor can be external to the automaton and provided when instantiating it.

>>> r,w=os.pipe()
>>> a=Example(debug=2,external_fd={"my_events":r})
>>> a.runbg()
DEBUG: ## state=[BEGIN]
>>> os.write(w, "hello")
5
Got ['hello']
DEBUG: I/O event [my_transition] not taken
>>> os.write(w, "go")
2
Got ['go']
DEBUG: I/O event [my_transition] taken to state [END]
DEBUG: switching from [BEGIN] to [END]
DEBUG: ## state=[END]
>>> a.run()
"job's done"

Decorator for actions

Actions are methods that are decorated by the return of ATMT.action function. This function takes the transition method it is bound to as first parameter and an optionnal priority prio as a second parameter. Default priority is 0. An action method can be decorated many times to be bound to many transitions.

class Example(Automaton):
    @ATMT.state(initial=1)
    def BEGIN(self):
        pass

    @ATMT.state(final=1)
    def END(self):
        pass

    @ATMT.condition(BEGIN, prio=1)
    def maybe_go_to_end(self):
        if random() > 0.5:
            raise self.END()
    @ATMT.condition(BEGIN, prio=2)
    def certainly_go_to_end(self):
        raise self.END()

    @ATMT.action(maybe_go_to_end)
    def maybe_action(self):
        print "We are lucky..."
    @ATMT.action(certainly_go_to_end)
    def certainly_action(self):
        print "We are not lucky..."
    @ATMT.action(maybe_go_to_end, prio=1)
    @ATMT.action(certainly_go_to_end, prio=1)
    def always_action(self):
        print "This wasn't luck!..."

The two possible outputs are

>>> a=Example()
>>> a.run()
We are not lucky...
This wasn't luck!...
>>> a.run()
We are lucky...
This wasn't luck!...

Methods to overload

Two methods are hooks to be overloaded.

The parse_args() method is called with arguments given at __init__() and run(). Use that to parametrize the behaviour of your automaton.

The master_filter() method is called each time a packet is sniffed and decides if it is interesting for the automaton. When working on a specific protocol, this is where you will ensure the packet belongs to the connection you are being part of, so that you do not need to make all the sanity checks in each transition.