rclone/librclone/README.md

librclone

This directory contains code to build rclone as a C library and the
shims for accessing rclone from C and other languages.

Note for the moment, the interfaces defined here are experimental
and may change in the future. Eventually they will stabilise and this
notice will be removed.

C

The shims are a thin wrapper over the rclone RPC.

The implementation is based on cgo; to build it you need Go and a GCC compatible
C compiler (GCC or Clang). On Windows you can use the MinGW ports, e.g. by installing
in a MSYS2 distribution (you may now install GCC in the newer
and recommended UCRT64 subsystem, however there were compatibility issues with previous
versions of cgo where, if not force rebuild with go build option -a helped, you had
to resort to the classic MINGW64 subsystem).

Build a shared library like this (change from .so to .dll on Windows):

go build --buildmode=c-shared -o librclone.so github.com/rclone/rclone/librclone

Build a static library like this (change from .a to .lib on Windows):

go build --buildmode=c-archive -o librclone.a github.com/rclone/rclone/librclone

Both the above commands will also generate librclone.h which should
be #included in C programs wishing to use the library (with some
exceptions).

The library will depend on libdl and libpthread on Linux/macOS, unless
linking with a C standard library where their functionality is integrated,
which is the case for glibc version 2.34 and newer.

You may add arguments -ldflags -s to make the library file smaller. This will
omit symbol table and debug information, reducing size by about 25% on Linux and
50% on Windows.

Note that on macOS and Windows the mount functions will not be available unless
you add additional argument -tags cmount. On Windows this also requires you to
first install the third party utility WinFsp,
with the "Developer" feature selected, and to set environment variable CPATH
pointing to the fuse include directory within the WinFsp installation
(typically C:\Program Files (x86)\WinFsp\inc\fuse). See also the
mount documentation.

On Windows, when you build a shared library, you can embed version information
as binary resource. To do that you need to run the following command before
the build command.

go run bin/resource_windows.go -binary librclone.dll -dir librclone

Documentation

For documentation see the Go documentation for:

• RcloneInitialize
• RcloneFinalize
• RcloneRPC
• RcloneFreeString

Linux C example

There is an example program ctest.c, with Makefile, in the ctest
subdirectory. It can be built on Linux/macOS, but not Windows without
changes - as described next.

Windows C/C++ guidelines

The official C example is targeting Linux/macOS, and will
not work on Windows. It is very possible to use librclone from a C/C++
application on Windows, but there are some pitfalls that you can avoid by
following these guidelines:

• Build librclone as shared library, and use run-time dynamic linking (see linking).
• Do not try to unload the library with FreeLibrary (see unloading).
• Deallocate returned strings with API function RcloneFreeString (see memory management).
• Define struct RcloneRPCResult, instead of including librclone.h (see include file).
• Use UTF-8 encoded strings (see encoding).
• Properly escape JSON strings, beware of the native path separator (see escaping).

Linking

Use of different compilers, compiler versions, build configuration, and
dependency on different C runtime libraries for a library and the application
that references it, may easily break compatibility. When building the librclone
library with MinGW GCC compiler (via go build command), if you link it into an
application built with Visual C++ for example, there will be more than enough
differences to cause problems.

Linking with static library requires most compatibility, and is less likely to
work. Linking with shared library is therefore recommended. The library exposes
a plain C interface, and by using run-time dynamic linking (by using Windows API
functions LoadLibrary and GetProcAddress), you can make a boundary that
ensures compatibility (and in any case, you will not have an import library).
The only remaining concern is then memory allocations; you should make sure
memory is deallocated in the same library where it was allocated, as explained
below.

Unloading

Do not try to unload the library with FreeLibrary, when using run-time dynamic
linking. The library includes Go-specific runtime components, with garbage
collection and other background threads, which do not handle unloading. Trying
to call FreeLibrary will crash the application. I.e. after you have loaded
librclone.dll into your application it must stay loaded until your application
exits.

Memory management

The output string returned from RcloneRPC is allocated within the librclone
library, and caller is responsible for freeing the memory. Due to C runtime
library differences, as mentioned above, it is not recommended to do
this by calling free from the consuming application. You should instead use
the API function RcloneFreeString, which will call free from within the
librclone library, using the same runtime that allocated it in the first
place.

Include file

Do not include librclone.h. It contains some plain C, golang/cgo and GCC
specific type definitions that will not compile with all other compilers
without adjustments, where Visual C++ is one notable example. When using
run-time dynamic linking, you have no use of the extern declared functions
either.

The interface of librclone is so simple, that all you need is to define the
small struct RcloneRPCResult, from librclone.go:

struct RcloneRPCResult {
    char* Output;
    int	Status;
};

Encoding

The API uses plain C strings (type char*, called "narrow" strings), and rclone
assumes content is UTF-8 encoded. On Linux systems this normally matches the
standard string representation, and no special considerations must be made. On
Windows it is more complex.

On Windows, narrow strings are traditionally used with native non-Unicode
encoding, the so-called ANSI code page, while Unicode strings are instead
represented with the alternative wchar_t* type, called "wide" strings, and
encoded as UTF-16. This means, to correctly handle characters that are encoded
differently in UTF-8, you will need to perform conversion at some level:
Conversion between UTF-8 encoded narrow strings used by rclone, and either ANSI
encoded narrow strings or wide UTF-16 encoded strings used in runtime function,
Windows API, third party APIs, etc.

Escaping

The RPC method takes a string containing JSON. In addition to the normal
escaping of strings constants in your C/C++ source code, the JSON needs its
own escaping. This is not a Windows-specific issue, but there is the
additional challenge that native filesystem path separator is the same as
the escape character, and you may end up with strings like this:

const char* input = "{"
"\"fs\": \"C:\\\\Temp\","
"\"remote\": \"sub/folder\","
"\"opt\": \"{\\\"showHash\\\": true}\""
"}";

With C++11 you can use raw string literals to avoid the C++ escaping of string
constants, leaving escaping only necessary for the contained JSON.

Example in golang

Here is a go example to help you move files :

func main() {
  librclone.Initialize()
    syncRequest: = syncRequest {
    SrcFs: "<absolute_path>",
    DstFs: ":s3,env_auth=false,access_key_id=<access>,secret_access_key=<secret>,endpoint='<endpoint>':<bucket>",
    }

    syncRequestJSON, err: = json.Marshal(syncRequest)
    if err != nil {
    fmt.Println(err)
    }
		
    out, status: = librclone.RPC("sync/copy", string(syncRequestJSON))
    fmt.Println("Got status : %d and output %q", status, out)
}

gomobile

The gomobile subdirectory contains the equivalent of the C binding but
suitable for using with gomobile
using something like this.

gomobile bind -v -target=android -javapkg=org.rclone github.com/rclone/rclone/librclone/gomobile

The command generates an Android library (aar) that can be imported
into an Android application project. Librclone will be contained
within libgojni.so and loaded automatically.

// imports
import org.rclone.gomobile.Gomobile;
import org.rclone.gomobile.RcloneRPCResult;

// initialize rclone
Gomobile.rcloneInitialize();

// call RC method and log response.
RcloneRPCResult response = Gomobile.rcloneRPC("core/version", "{}");
Log.i("rclone", "response status: " + response.getStatus());
Log.i("rclone", "output: " + response.getOutput());

// Clean up when finished.
Gomobile.rcloneFinalize();

This is a low level interface - serialization, job management etc must
be built on top of it.

iOS has not been tested (but should probably work).

Further docs:

• gomobile main website
• gomobile wiki
• go issue #16876 where the feature was added
• gomobile design doc for extra details not in the docs.

python

The python subdirectory contains a simple Python wrapper for the C
API using rclone linked as a shared library with ctypes.

You are welcome to use this directly.

This needs expanding and submitting to pypi...

Rust

Rust bindings are available in the librclone crate: https://crates.io/crates/librclone

PHP

The php subdirectory contains how to use the C library librclone in php through foreign
function interface (FFI).

Useful docs:

• PHP / FFI

TODO

• Async jobs must currently be cancelled manually at the moment - RcloneFinalize doesn't do it.
• This will use the rclone config system and rclone logging system.
• Need examples showing how to configure things,