Bug #44351 C++ rsync wrapper for atrt
Submitted: 18 Apr 2009 7:49 Modified: 3 Feb 2010 6:43
Reporter: Magnus Blåudd Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S4 (Feature request)
Version:mysql-5.1-telco-7.0 OS:Any
Assigned to: jack andrews CPU Architecture:Any

[18 Apr 2009 7:49] Magnus Blåudd
Description:
In order to be able to port atrt to Windows we want to remove its dependency
on external shell scripts. The main functionality of those scripts are creating
directories, removing files and copying data between hosts. The scripts always work locally and then invoke rsync to send the directories and files to the test hosts. We still want to depend on rsync but make it possible to control it directly from C++ through a rsync wrapper class. That way we can replace the shell scripts with some C++ 'mkdir' and 'rmdir' plus use the rsync through the C++ wrapper.

How to repeat:
.

Suggested fix:
storage/ndb/test/run-test/RSync.hpp:

class RSync {
  BaseString m_rsync;
  BaseString m_path;
public:
  /*
    Create  Rsync class, store the path for later
  */
  Rsync(const char* path = NULL) : m_path(path) {}
  
  
  /*
    Search for rsync, using "m_path" if set else assume
    it's in $PATH. Detect version and check that version is "high enough"
    Return false if not found and print error to g_logger, the calling program
    will then most likely exit.
    See "client/mysql_upgrade.c" or "client/mysqltest.cc" for examples how to run external program
  */
  bool init();

  /*
    Call rsync to perform requested sync operation
  */
  bool to_remote(const char* lpath, const char* rhost, const char* rpath);
  bool to_local(const char* rhost, const char* rpath, const char* lpath);

}

Implementation in ndb/test/run-test/RSync.cpp
[22 Apr 2009 1:49] jack andrews
would a .bat file be more appropriate?

is cygwin so bad?

why do we need to write a wrapper, can't we just use rsync directly?

if we need to write a wrapper, why not do it in perl?

would need to handle options to rsync, too:

$ grep rsync *
README.ATRT:!-- deploy binaries and libraries to hosts, rsync
atrt-gather-result.sh:  rsync -a --exclude='BACKUP' --exclude='ndb_*_fs' "$1" .
atrt-setup.sh:rsync -a --delete --force --ignore-errors $2 $1:$3
autotest-boot.sh:# create the database to be rsynced     #
main.cpp:do_rsync(const char *dir, const char *dst)
main.cpp:  g_logger.info("rsyncing %s to %s", dir, dst);
main.cpp:    g_logger.critical("Failed to rsync %s to %s", dir, dst);
main.cpp:    if (!do_rsync(g_basedir, config.m_hosts[i]->m_hostname.c_str()))
main.cpp:    if (!do_rsync(g_prefix, config.m_hosts[i]->m_hostname.c_str()))
main.cpp:    if (g_prefix1 && !do_rsync(g_prefix1, config.m_hosts[i]->m_hostname
.c_str()))
ndb-autotest.sh:# create the database to be rsynced      #
ndb-autotest.sh:        (cd $install_dir; ./bin/mysql_install_db) # This will be
 rsynced to all
ndb-autotest.sh:      rsync -a --delete --force --ignore-errors $install_dir/ $i
:$install_dir
ndb-autotest.sh:          echo "$i failed during rsync, excluding"
upgrade-boot.sh:# create the database to be rsynced      #
[28 Apr 2009 16:40] Magnus Blåudd
Primarily "we want to remove its dependency on external shell scripts". In addition we make atrt more self contained (i.e easier to install and debug) and it will also be possible to better handle error and print debug messages if anything goes wrong(i.e we get better control).

It won't need to handle any options to rsync to start with.  The 'to_remote' function would more or less be equvivalent to "rsync -a --delete --force --ignore-errors $2 $1:$3"
[3 Feb 2010 6:42] jack andrews
we used a different approach to porting autotest to windows.