Lua-cURL

Lua-cURL is aiming for a full-fledged libcurl binding (easy/multi/share interface) to the functionality of Lua

Download as .zip Download as .tar.gz View on GitHub
Original Author (before of 4Q-2011): Jürgen Hötzel
Current Lead Developer: Vadim A. Misbakh-Soloviov
Contact: http://mva.name/

Contents

Introduction

This project is not a fork of LuaCURL, which is a direct mapping of parts of the libcurl-easy interface.

The intent of Lua-cURL is to adapt the

of libcurl to the functionality of Lua (for example by using iterators instead of callbacks when possible).

Installation

We now using CMake to configure, build and install. Just run:

cmake ${CMAKE_ARGUMENTS} . && make ${MAKE_ARGUMENTS} install

Where CMAKE_ARGUMENTS may contain one or all of the following (in one space-delimited string or bash's array, without comments):

-DCMAKE_C_FLAGS="-flto"
# to use LTO (Link-Time Optimisations). Requires GCC>=4.6

-DUSE_LUAJIT=ON
# to use LuaJIT includes instead of 'C' Lua library. Requires LuaJIT>=2.0

-DCMAKE_LINKER="/usr/bin/ld.gold"
# To use 'gold' linker. Requires newer binutils.

-DUSE_LUA52=ON
# to install for Lua 5.2. Requires Lua>=5.2

And MAKE_ARGUMENTS may also contain one or all of the following (in one space-delimited string or bash's array, without comments):

-jN
# Where N = Number of processor cores * Number of processors + 1,
# to build in N threads, to speedup compilation (by default, -j1)

DESTDIR=""
# Where  can be either absolute or relative prefix-path to the place,
# where you want to install Lua-cURL (by default, DESTDIR=/)

On some systems, CMake may fail to include the right lua header files. This error can be identified at runtime, after requiring the cURL module, by the error message "undefined symbol". In that case the module must be recompiled specifying the include path for lua header files. This can be done with the LUA_DIR environment variable. E.g. in a posix-compliant shell:

LUA_DIR="/usr/include/lua5.1" cmake ${CMAKE_ARGUMENTS}

Easy interface

cURL.easy_init()
returns a new easy handle.
cURL.version_info()
returns a table containing version info and features/protocols sub table
easy:escape(string)
return URL encoded string
easy:unescape(string)
return URL decoded string
easy:setopt*(value)
libcurl properties an options are mapped to individual functions:
  • easy:setopt_url(string)
  • easy:setopt_verbose(number)
  • easy:setopt_proxytype(string)
  • easy:setopt_share(share) (See: share)
  • ...
easy:perform(table)
Perform the transfer as described in the options, using an optional callback table.The callback table indices are named after the equivalent cURL callbacks:
  • writefunction = function(str)
  • readfunction = function()
  • headerfunction = function(str)
easy:post(table)

Prepare a multipart/formdata post. The table indices are named after the form fields and should map to string values:

{field1 = value1,
 field2 = value1}

or more generic descriptions in tables:

{field1 = {file="/tmp/test.txt",
           type="text/plain"},
{field2 = {file="dummy.html",
           data="<html><bold>bold</bold></html>,
           type="text/html"}}

Example 1: Fetch the example.com homepage

require("cURL")

-- open output file
f = io.open("example_homepage", "w")

c = cURL.easy_init()
-- setup url
c:setopt_url("http://www.example.com/")
-- perform, invokes callbacks
c:perform({writefunction = function(str)
                f:write(str)
                 end})

-- close output file
f:close()
print("Done")

Example 2: "On-The-Fly" file upload

-- simple "On the fly" fileupload

require("cURL")

c=cURL.easy_init()
c:setopt_url("ftp://ftptest:secret0815@targethost/file.dat")
c:setopt_upload(1)
count=0
c:perform({readfunction=function(n)
               count = count + 1
               if (count < 10)  then
                  return "Line " .. count .. "\n"
               end
               return nil
            end})
print("Fileupload done")

Example 3: "Posting" data

require("cURL")

c = cURL.easy_init()

c:setopt_url("http://localhost")
postdata = {
   -- post file from filesystem
   name = {file="post.lua",
       type="text/plain"},
   -- post file from data variable
   name2 = {file="dummy.html",
        data="<html><bold>bold</bold></html>",
        type="text/html"}}
c:post(postdata)
c:perform()

print("Done")

Multi interface

cURL.multi_init()
returns a new multi handle
multi:add_handle(easy)
add an easy handle to a multi session
multi:perform()

returns an iterator function that, each time it is called, returns the next data, type and corresponding easy handle:

data:
data returned by the cURL library
type
type of data returned ("header" or "data")
easy
corresponding easy handle of the data returned

Example 1: "On-The-Fly" XML parsing

-- use LuaExpat and Lua-CuRL together for On-The-Fly XML parsing
require("lxp")
require("cURL")

tags = {}
items = {}

callback = {}

function callback.StartElement(parser, tagname)
   tags[#tags + 1] = tagname
   if (tagname == "item") then
      items[#items + 1] = {}
   end
end

function callback.CharacterData(parser, str)
   if (tags[#tags -1] == "item") then
      --we are parsing a item, get rid of trailing whitespace
      items[#items][tags[#tags]] = string.gsub(str, "%s*$", "")
   end
end
function callback.EndElement(parser, tagname)
   --assuming well formed xml
   tags[#tags] = nil
end

p = lxp.new(callback)

-- create and setup easy handle
c = cURL.easy_init()
c:setopt_url("http://www.lua.org/news.rss")

m = cURL.multi_init()
m:add_handle(c)

for data,type in m:perform() do
   -- ign "header"
   if (type == "data") then
      assert(p:parse(data))
   end
end

--finish document
assert(p:parse())
p:close()

for i, item in ipairs(items) do
   for k, v in pairs(item) do
      print(k,v)
   end
   print()
end

Share interface

cURL.share_init()
returns a new share handle
share:setopt_share(string)
specifies the type of data that should be shared ("DNS" or "COOKIE")

Since Lua is single-threaded, there is no mapping for the lock options.

Example 1: Share Cookie date across easy handles

-- Cookie data will be shared across the easy handles to do an authorized download
require("cURL")

-- create share handle (share COOKIE and DNS Cache)
s = cURL.share_init()
s:setopt_share("COOKIE")
s:setopt_share("DNS")

-- create first easy handle to do the login
c = cURL.easy_init()
c:setopt_share(s)
c:setopt_url("http://targethost/login.php?username=foo&password=bar")

-- create second easy handle to do the download
c2 = cURL.easy_init()
c2:setopt_share(s)
c2:setopt_url("http://targethost/download.php?id=test")

--login
c:perform()

--download
c2:perform()

Appendix

Using SSL

The cert bundle distributed with cURL may be out of date and cannot validate many certificates. You can supply a different PEM cert bundle by using easy:setopt_cainfo(string). Also, it is a shell script, writen by original author (Jürgen), that can be used for converting the cacert keystore distributed with the Java Runtime Environment to PEM. This script currently distributed in «examples» folder in repository