--- [ Nmap/Ncrack ] ---

Nmap

ServiceScan

Nsock

Ncrack

Initial Draft

Core Engine

Command Line

Status Reports 2009

Status Reports 2010

GSoC/Ncrack presentation

Ncrack Developer's Guide

Network Exploitation with Ncrack

Network Exploitation with Ncrack video at AthCon2011

 
 
  Ncrack initial draft specifications and thoughts
  ==================================================
  
  As you have already read from Fyodor's mail
  ( http://seclists.org/nmap-dev/2009/q2/0238.html ), this year's SoC will involve
  making a new network authentication cracker under the name Ncrack.
  
  Fyodor and I had a discussion some days ago and concluded that a solid plan is
  needed before starting to code. So in this post, I will outline my thoughts on
  some aspects of a possible architecture of Ncrack.
  
  First of all, since we should get all the existing tools' best ideas and maybe
  the most representative of them is thc-hydra, I have been studying its source
  code to see how its internals work. Below follow some notes I took. You can skip
  this part if you are not that interested in thc-hydra, but I personally think
  that getting a good grasp on one of perhaps the most competitive open source
  authentication cracking tools out there, can certainly be of use and thus
  suggest that you actually do read it.
  
  
  **************************** thc-hydra ***********************************
  
  How thc-hydra works
  =====================
  
  
  Hydra is a parallelized network authentication cracker written by Van Hauser of
  tch. It is based on a modularized architecture, where each module corresponds to
  a certain protocol service. Each service module belongs to its own independent
  .c file. The user defines what kind of service he wants to attack and hydra
  spawns the equivalent module a number of times equal to the number of concurrent
  processes defined (-t option).
  
  
  Architecture
  =============
  
  -- General structures
  
   hydra_arm: represents child information (its pid_t, a pair of unix domain
   socket descriptors used for communicating with parent, the current username &
   password to be tested etc)
  
   hydra_target: represents a target - a number equal to the list of IPs defined
   in the input file (-M option) will be created:
  
   /-----------------------------------------------------------\
  
  
      fill_mem(servers_ptr, ifp, 0);
      sizeservers = sizeinfile; tmpptr = servers_ptr;
      for (i = 0; i < countinfile; i++) {
          hydra_targets[i] = malloc(sizeof(hydra_target));
          memset(hydra_targets[i], 0, sizeof(hydra_target));
          hydra_targets[i]->target = tmpptr;
          while (*tmpptr != 0)
              tmpptr++;
          tmpptr++;
          hydra_targets[i]->login_ptr = login_ptr;
          hydra_targets[i]->pass_ptr = pass_ptr;
          hydra_targets[i]->max_use_count =
          hydra_options.max_use;
      } 
  
   \-----------------------------------------------------------/
  
   login_ptr & pass_ptr are memory regions that have been filled with the
   username/password lists from the corresponding files (-L,-P,-C)
  
  
  -- Parent-child communication
  
  The communication between the process mother and the modules happens through the
  use of UNIX domain sockets (AF_UNIX). hydra_spawn_arm() creates a pair of
  unnamed unix domain socket descriptors using socketpair(), forks a new child
  which executes the selected service module and passes the one copy of the
  descriptor as argument. Then the child writes and reads to that descriptor to
  inform the mother process of its status (e.g it failed to connect to the service
  / it found a new password / it exited due to an error etc). The mother process
  can also write to that descriptor to pass information such as a new
  username/password pair (e.g write(hydra_arms[arm_no]->sp[0], buf, buflen);)
  Possible child values are:
  
  
     /* Valid Results:
      * n - mother says to itself that child requests next login/password pair
      * N - child requests next login/password pair
      * Q - child reports that it is quitting
      * C - child reports connect error (and is quitting)
      * E - child reports protocol error (and is quitting)
      * f - child reports that the username does not exist
      * F - child reports that it found a valid login/password pair
      * and requests next pair. Sends login/pw pair with next msg!
      */ 
  
  
  -- Main process loop
  
  The mother process' main loop uses select() to go through the unix domain socket
  descriptors of its children and test if any of them has something new to report.
  
  /-----------------------------------------------------------\
  
   /* this is the big function which starts the attacking children, feeds
    * login/password pairs, etc.! */
    while (hydra_brains.finished < hydra_brains.targets && error == 0) {
    FD_ZERO(&fdreadarms);
  
      ...
  
      my_select(max_fd+1,&fdreadarms,NULL,NULL,0,200000);
      for (arm_no = 0; arm_no < hydra_options.tasks; arm_no++) { 
      ...
  
   \-----------------------------------------------------------/
  
  Additionally it spawns new childs using hydra_spawn_arm() as needed.
  
  
  -- helper functions
  
  fill_mem(): will read the input password file and copy each username/password to
  the (already) allocated pointer passed to it.
  
  hydra_restore_read(): will restore a session from file hydra_restore_write():
  will save current session to file for later use
  
  
  Parallelization 
  =================
  
  The parallelization is achieved through process forking. The user can define the
  maximum number of concurrent tasks (processes) by using the -t option. 
  
  
  
  Interesting Points
  ===================
  
  Failed connection attempts 
  ---------------------------
  
  What happens if a child fails to connect to a particular server? Hydra uses the
  notion of fail_count which keeps a note of how many attempts it tried to connect
  to the particular server (it is a member of the hydra_target struct).  Every
  time a child reports a 'C' or 'E' message (connect error and protocol error)
  will subsequently quit and the mother will increase the fail_count counter of
  the target. If the fail_count reaches the MAXFAIL threshold (by default it is 3)
  then hydra will never try again for the particular target (it will be marked as
  'done' in global list hydra_targets). If this doesn't happen, then the target's
  hydra_arm will set the 'redo' variable to try again later.
  
  This is an interesting point, since we usually assume that the user provides a
  list of IP addresses that have already been scanned for a particular service
  (using Nmap of course!) and thus the port is open. However, firewalls using
  dynamic rulesets may block us if we surpass a certain amount of connections
  inside a certain time period.
  
  
  Next target selection 
  ---------------------- 
  
  Hydra selects, at each round inside the main loop, the target that the next
  spawned child will attempt to attack. The algorithm for the decision is firstly
  based on choosing the target that is attacked by the smallest number of 
  children and secondly (in case of draw before) on the smallest fail count
  (as mentioned above).
  
  
  Module engine
  ---------------
  
  Hydra's module engine is simple in general. Each module gets called by a
  function with name service_<service> that has the following prototype:
  
  void service_<service>(unsigned long int ip, int sp, unsigned char options, char
  *miscptr, FILE * fp, int port)
  
  ip -> target ip address sp -> unix domain socket descriptor for parent
  communication options -> set to OPTION_SSL if we want to use ssl miscptr ->
  pointer to additional options if module requires them fp -> file pointer to
  output for successfully found username/password reports (can be stdout or a
  file) port -> defined for non-standard listening service ports 
  
  Every network operation takes place inside each module using custom network
  handlers defined in hydra-mod.c
  
  
  Hydra-arms 
  ----------- 
  OK that should be heads instead of arms, according to Greek mythology on hydra.
  
  
  **************************** thc-hydra end ********************************
  
  
  
  Ncrack proposed architecture
  ==============================
  
  Ncrack will be based on the Nsock library, which is, as you all know, a parallel
  sockets library that is currently used by Nmap's service_scan engine, NSE and
  nmap_dns.cc. I have taken some notes on some aspects of Nsock at my site's wiki
  here: http://sock-raw.org/wiki/Nmap/Nsock
  
  Nsock uses callback handlers for all main network operations (connect, read,
  write). Connect(2) is done using non-blocking sockets that adds to the speed.
  nsock_loop() is used to to go through all the socket descriptors ( using
  select() - optionally with a timeout ) and upon a new event(e.g new data
  arriving from network) call the corresponding callback handler that has been
  previously registered with each kind of operation.  Nsock's way of working is
  fundamental in designing a proper communication API between Ncrack's core
  engine and the separate modules that will be service/protocol specific. The 
  modules will ideally need to be simple and small, while the core engine will be
  responsible for all network operations, option parsing, timing algorithms and
  communicating with the modules.
  
  
  Engine - Module API
  ====================
  
  To keep modules as simple as possible, we could build them as state machines.
  Ideally, they should be a certain sequence of specific steps they must carry
  through to make the authentication. Communication with the main engine will
  be done through a generic struct that will hold all necessary information.
  Example: 
    struct module_data { int state, int protocol, unsigned short port,
    char *username, char *password, ... }
  
  module_data will have to be 'connected' with one of nsock_pool's sockets.
  If I am correct, struct msiod (which is then cast to nsiod) from
  nsock/src/nsock_internal.h has a void *userdata member which could be used for
  this kind of message passing.
  
  service_scan.cc is a concrete example of using nsock and some ideas can be
  used from its codebase. Thus we could have the main cracking handler in
  our core engine:
  
  ncrack_crack() {
  
   ... variables ...
  
   create nsock pool
   sendSomeServiceProbes()
  
   nsock_loop()
   ...
  
  }
  
  
  sendSomeServiceProbes()
  {
  
    ...
    while () {
      nsock_connect_tcp/udp(..., connect_handler)
    }
  
  }
  
  connect_handler()
  {
    ...
    call_module(<service_name>, (struct *module_data) nsiod->userdata)
  }
  
  
  Thus the connection handler will invoke the generic call_module handler
  and will pass to it the module_data struct (which at first will be
  initialized to certain values)
  Then call_module, will invoke the proper module by parsing the <service_name>.
  An example module then would be like this:
  
  enum { READ_OP, WRITE_OP };
  
  http_module(nsiod *io) {
  
    ...
    struct *module_data = (struct *module_data) io->userdata
    switch (module_data->state) {
      case INIT:
        foo;
        module_data->state = NEXT;
        module_data->operation = READ_OP; // or write_op or anything
        ncrack_request_operation(io);
      case NEXT:
        bar;
        ...
      case FINI:
        sha;
      }
  }
  
  
  ncrack_request_operation() (from the core_engine) will register a new nsock
  event based on the operation requested on the module_data struct and will also
  pass the module_data back to the event's callback handler which will
  subsequently reinvoke the module with the module_data that now has the new
  state. The new state will now make the module follow a different code path
  accordingly and so on.
  
  I assume, though I am not entirely certain, that all protocols can be dissected
  to use a state machine like the above. If, however, you believe that some
  protocols are that quirky that the above model will overcomplicate things
  instead of simplifying them, please say so.
  
  Now let's move on to some other things.
  
  
  Timing options
  ===============
  
  One of the most importan features of Nmap are its timing-tuninig options.
  Since many users are already accustomed with Nmap's option conventions, I
  suggest that as long as the option names don't provoke confusion, that we
  keep them as is. For example, take a look at the following options (from nmap
  man page)
  
  --min-hostgroup numhosts; --max-hostgroup numhosts 
  (Adjust parallel scan group sizes) .
  
  In nmap, these two options denote the number of hosts that will be scanned in
  parallel. Ncrack will support cracking a multitude of hosts at the same time,
  so having such an option with the same name will be useful (we are referring
  to exactly the same thing)
  
  
  --min-parallelism numprobes; --max-parallelism numprobes
  (Adjust probe parallelization) .
  
  In nmap, this denotes the total number of outstanding probes for a hostgroup. 
  In ncrack, this will denote the total number of active connections.
  
  
  --max-retries numtries
  (Specify the maximum number of port scan probe retransmissions) .
  
  In nmap, this specifies how many probes will hit the same port at the same
  host in case we haven't received an answer yet.
  In ncrack, this can specify how many retries to connect to a certain service
  before marking the host service as dead (see thc-hydra failed connection
  attempts above).
  
  
  --scan-delay time; --max-scan-delay time (Adjust delay between probes) .
  
  This may need a name-change but essentially, it will have the same effect.
  It will limit the number of probes (as in any write(2) network operation)
  per second per host.
  
  
  
  Target Specification
  ====================
  
  Target specification should also be similar to that of Nmap's. The user
  should be able to either give a list of command-line IP addresses or
  hostnames as well as give an input filename with the option -iL.
  
  Implementing options like --exclude or --excludefile and CIDR notation /
  whole network range specification might at first sound like an overhead,
  since blindly specifying hosts might be a rare case (users are supposed
  to usually specify a list of servers that have previously scanned and thus
  know that have open ports on the specified service - Ncrack will have no
  portscanning functionality) but nevertheless they might be of some use.
  
  Being able to use Nmap's output automatically is another useful and
  possible feature. I think parsers for nearly every Nmap's output format
  are already available (OK apart from -oS I guess :) ) so that won't be
  much of a problem. However, each Ncrack instance will be supposed to
  scan only one particular kind of service and consequently tha only useful
  parsable information will be the host IP addresses that have that
  particular port marked as open.
  
  Finally, name resolution can happen with Nmap's already existing parallel
  dns resolver.
  
  Password policy
  ================
  
  thc-hydra has a separate tool called pw-inspector that parses a password
  file and enforces certain policies so that passwords that do not meet
  the criteria are stripped from the output result. We could make something
  similar or integrate it inside Ncrack. But having it as in-built will
  result in more option overhead.
  
  All the above are just an initial rough draft. I will probably post an
  update soon including more aspects. Until then, I would be glad to hear
  your thoughts and feedback on the general design, especially the engine
 - module API.