diff --git a/lib/actors/README.docdown.md b/lib/actors/README.docdown.md new file mode 100644 index 0000000000..ece57aef6f --- /dev/null +++ b/lib/actors/README.docdown.md @@ -0,0 +1,104 @@ +# `actors` - Nit Actor Model + +[[toc: actors]] + +This group introduces the [[actors::actors | text: `actors`]] module which contains the abstraction of a Nit Actor Model, +based on Celluloid (https://github.com/celluloid/celluloid). + +Example from `actors::chameneosredux`: + +[[code: actors::chameneosredux]] + +## What is an actor ? + +[[uml: actors | format: svg, mentities: actors::actors_simple;actors::actors_mandelbrot;actors::actors_thread_ring;actors::actors_simple_simulation;actors::actors_agent_simulation;actors::actors_chameneosredux;actors::actors_fannkuchredux;actors::actors]] + +An actor is an entity which receives [[actors::Message | text: messages]] and does some kind of computation based on it. +An actor has a [[actors::Mailbox | text: mailbox]] in which it receives its messages, and process them one at a time. + +Example from `actors::simple_simulation`: + +[[code: actors::simple_simulation]] + +[[features: actors | mentities: actors::actors_simple;actors::actors_mandelbrot;actors::actors_thread_ring;actors::actors_simple_simulation;actors::actors_agent_simulation;actors::actors_chameneosredux;actors::actors_fannkuchredux;actors::actors]] + +## `actor` annotation + +The [[actors | text: `actors`]] module introduces the annotation `actor` which is to be used on classes. +This annotation transform a normal Nit class into an actor. + +In practice, it adds a new property `async` to the annotated class. +When using `async` on your annotated class, this means that you want your calls to be asynchronous, +executed by the [[actors::Actor | text: actor]]. + +For instance, if you call `a.async.foo` and `foo` doesn't have a return value, it will send +a message to the mailbox of the actor attached to `a` which will process it asynchronously. + +On the other hand, if you call `a.async.bar` and `bar` returns an`Int`, it will still send +a message to the actor, but you'll get a [[actors::Future | text: `Future[Int]`]] to be able to retrieve the value. +When using `join` on the future, the calling thread will wait until the value of the future is set. + +Example from `actors::mandelbrot`: + +[[code: actors::mandelbrot]] + +## Managing actors + +When you annotate a class with `actor` and create an instance of it with `new`, the actor is not +automatically created (which means you can completely skip the use of the actors if you +don't need them for a specific program). + +The `async` added property is actually created lazily when you use it. + +Actors are not automatically garbage collected, but you have solutions to terminate them +if you need to. For this, you need to use the `async` property of your annotated class : + +* `async.terminate` sends a shutdown message to the actor telling him to stop, so he'll finish + processing every other messages in his mailbox before terminating properly. Every other messages sent + to this actor after he received the shutdown message won't be processed. +* `async.terminate_now` sends a shutdown message too, but this time it places it first, so + if the actor is processing one message now, the next one will be the shutdown message, discarding + every messages in its mailbox. +* `async.wait_termination` wait for the actor to terminate properly. This call is synchronous. +* `async.kill`. If you really need this actor to stop, without any regards of what he was doing + or in which state he'll leave the memory, you can with this call. it's synchronous but not really + blocking, since it's direcly canceling the native pthread associated to the actor. + +For now, there isn't any mecanism to recreate and actor after it was terminated. +Sending messages after terminating it results in unspecified behaviour. + +Example from `actors::simple`: + +[[code: actors::simple]] + +## Waiting for all actors to finish processing + +Let's imagine you create a whole bunch of actors and make them do things asynchronously from the main thread. +You don't want your program to exit right after giving work to your actors. +To prevent that, we added a mecanism that waits before all your actors finished all their messages +before quitting. + +It's materialized by the `active_actors` property added to `Sys` which is a `ReverseBlockingQueue`. +In short, the `is_empty` method on this list is blocking until the list is effectively empty. +When every actors finished working, and we're sure they won't even send another message to another +actor, `active_actors` is empty. + +You can use this property as a mean of synchronisation in some specific cases (for example if you're +using actors for fork/join parallelism instead of concurrency). + +Example from `actors::fannkuchredux`: + +[[code: actors::fannkuchredux]] + +## Examples + +You can find example of differents small programs implemented with Nit actors in the `examples` +directory. For a really simple example, you can check `examples/simple`. + +Example from `actors::agent_simulation`: + +[[code: actors::agent_simulation]] + +## Authors + +This project is maintained by [[ini-maintainer: actors]]. diff --git a/lib/actors/README.md b/lib/actors/README.md index f99f1aab0f..a0bb65d809 100644 --- a/lib/actors/README.md +++ b/lib/actors/README.md @@ -1,30 +1,347 @@ -# Nit Actor Model +# `actors` - Nit Actor Model -This group introduces the `actors` module which contains the abstraction of a Nit Actor Model, +* [What is an actor ?](#What-is-an-actor-?) +* [`actor` annotation](#`actor`-annotation) +* [Managing actors](#Managing-actors) +* [Waiting for all actors to finish processing](#Waiting-for-all-actors-to-finish-processing) +* [Examples](#Examples) +* [Authors](#Authors) + +This group introduces the [`actors`](actors::actors) module which contains the abstraction of a Nit Actor Model, based on Celluloid (https://github.com/celluloid/celluloid). +Example from `actors::chameneosredux`: + +~~~ +# Example implemented from "The computer Language Benchmarks Game" - Chameneos-Redux +# http://benchmarksgame.alioth.debian.org/ +# +# Complete description of the chameneos-redux : +# https://benchmarksgame.alioth.debian.org/u64q/chameneosredux-description.html#chameneosredux +module chameneosredux is example, no_warning("missing-doc") + +import actors + +class Creature + actor + var place: MeetingPlace + var color: Int + var id: Int + var count = 0 + var samecount = 0 + + fun run do + loop + var p = place.meet(id, color) + if p == null then break + color = p.color + if p.sameid then samecount += 1 + count += 1 + end + end + + fun to_string: String do return count.to_s + " " + numbers[samecount] +end + +class Pair + var sameid: Bool + var color: Int +end + +class MeetingPlace + var meetings_left: Int + var firstcolor: nullable Int + var firstid: Int = 0 + var current: Future[Pair] is noinit + + private var mutex = new Mutex + + fun meet(id, c: Int): nullable Pair do + var new_pair = new Future[Pair] + mutex.lock + if meetings_left == 0 then + mutex.unlock + return null + else + if firstcolor == null then + firstcolor = c + firstid = id + current = new Future[Pair] + else + var color = complements[c][firstcolor.as(not null)] + current.set_value(new Pair(id == firstid, color)) + firstcolor = null + meetings_left -= 1 + end + new_pair = current + end + mutex.unlock + return new_pair.join + end +end + +redef class Sys + fun blue: Int do return 0 + fun red: Int do return 1 + fun yellow: Int do return 2 + var numbers = ["zero", "one", "two", "three", "four", "five", "six", "seven", "eight", "nine"] + var colors = ["blue", "red", "yellow"] + # Matrix for complementing colors + var complements: Array[Array[Int]] = [[0, 2, 1], + [2, 1, 0], + [1, 0, 2]] + +end + +fun print_all_colors do + print_colors(blue, blue) + print_colors(blue, red) + print_colors(blue, yellow) + print_colors(red, blue) + print_colors(red, red) + print_colors(red, yellow) + print_colors(yellow, blue) + print_colors(yellow, red) + print_colors(yellow, yellow) +end + +fun print_colors(c1, c2: Int) do + print colors[c1] + " + " + colors[c2] + " -> " + colors[complements[c1][c2]] +end + +fun get_number(n: Int): String do + var str = "" + var nstr = n.to_s + for c in nstr do + str += " " + numbers[c.to_i] + end + return str +end + +fun work(n, nb_colors : Int ) do + var place = new MeetingPlace(n) + var creatures = new Array[Creature] + for i in [0..nb_colors[ do + printn " " + colors[i % 3] + creatures[i] = new Creature(place, i % 3, i) + end + print "" + + for c in creatures do c.async.run + + active_actors.wait + + var total = 0 + for c in creatures do + print c.to_string + total += c.count + end + + print get_number(total) + print "" + + for c in creatures do + c.async.terminate + c.async.wait_termination + end +end + +var n = if args.is_empty then 600 else args[0].to_i + +print_all_colors +print "" + +work(n, 3) +work(n, 10) +~~~ + ## What is an actor ? -An actor is an entity which receives messages and does some kind of computation based on it. -An actor has a mailbox in which it receives its messages, and process them one at a time. +![Diagram for `actors`](uml-actors-7.svg) + +An actor is an entity which receives [messages](actors::Message) and does some kind of computation based on it. +An actor has a [mailbox](actors::Mailbox) in which it receives its messages, and process them one at a time. + +Example from `actors::simple_simulation`: + +~~~ +# Using `agent_simulation` by refining the Agent class to make +# a multi-agent simulation where every agent know each other +# The steps consist of each agent greeting each other, and +# waiting for every other agent to respond before notifying +# to the `ClockAgent` that they finished their step. +module simple_simulation is example + +import agent_simulation +redef class Agent + var others = new Array[Agent] + var count = 0 + + fun greet(message: String, other: Agent) do other.async.greet_back("Hello back !") + + fun greet_back(message: String) do + count -= 1 + if count == 0 then end_step + end + + redef fun do_step do + for o in others do + o.async.greet("Hello !", self) + count += 1 + end + end +end + +var nb_steps = 0 +var nb_agents = 0 +if args.is_empty or args.length != 2 then + nb_steps = 10 + nb_agents = 10 +else + nb_steps = args[0].to_i + nb_agents = args[1].to_i +end + +var agents = new Array[Agent] +for i in [0..nb_agents[ do agents.add(new Agent) +for a in agents do for b in agents do if a != b then a.others.add(b) +clock_agent = new ClockAgent(nb_steps, agents) +clock_agent.async.do_step +~~~ + +* `actors` - Abstraction of the actors concepts +* `actors_agent_simulation` - This file is generated by nitactors (threaded version) +* `actors_chameneosredux` - This file is generated by nitactors (threaded version) +* `actors_fannkuchredux` - This file is generated by nitactors (threaded version) +* `actors_mandelbrot` - This file is generated by nitactors (threaded version) +* `actors_simple` - This file is generated by nitactors (threaded version) +* `actors_simple_simulation` - This file is generated by nitactors (threaded version) +* `actors_thread_ring` - This file is generated by nitactors (threaded version) ## `actor` annotation -The `actors` module introduces the annotation `actor` which is to be used on classes. +The [`actors`](actors) module introduces the annotation `actor` which is to be used on classes. This annotation transform a normal Nit class into an actor. In practice, it adds a new property `async` to the annotated class. When using `async` on your annotated class, this means that you want your calls to be asynchronous, -executed by the actor. +executed by the [actor](actors::Actor). For instance, if you call `a.async.foo` and `foo` doesn't have a return value, it will send a message to the mailbox of the actor attached to `a` which will process it asynchronously. On the other hand, if you call `a.async.bar` and `bar` returns an`Int`, it will still send -a message to the actor, but you'll get a `Future[Int]` to be able to retrieve the value. +a message to the actor, but you'll get a [`Future[Int]`](actors::Future) to be able to retrieve the value. When using `join` on the future, the calling thread will wait until the value of the future is set. +Example from `actors::mandelbrot`: + +~~~ +# Example implemented from "The computer Language Benchmarks Game" - Mandelbrot +# http://benchmarksgame.alioth.debian.org/ +# +# Complete description of mandelbrot : +# https://benchmarksgame.alioth.debian.org/u64q/mandelbrot-description.html#mandelbrot +module mandelbrot is example, no_warning("missing-doc") + +import actors + +class Worker + actor + + fun get_byte(x, y: Int): Int do + var res = 0 + for i in [0..8[.step(2) do + var zr1 = crb[x + i] + var zi1 = cib[y] + + var zr2 = crb [x + i + 1] + var zi2 = cib[y] + + var b = 0 + for j in [0..nb_rounds[ do + var nzr1 = zr1 * zr1 - zi1 * zi1 + crb[x+i] + var nzi1 = zr1 * zi1 + zr1 * zi1 + cib[y] + zr1 = nzr1 + zi1 = nzi1 + + var nzr2 = zr2 * zr2 - zi2 * zi2 + crb[x + i + 1] + var nzi2 = zr2 * zi2 + zr2 * zi2 + cib[y] + zr2 = nzr2 + zi2 = nzi2 + + if zr1 * zr1 + zi1 * zi1 > 4.0 then b = b | 2 + if zr2 * zr2 + zi2 * zi2 > 4.0 then b = b | 1 + if b == 3 then break + end + res = (res << 2) + b + end + return res ^-1 + end + + fun put_line(y: Int, line: Array[Int]) do + for i in [0..line.length[ do line[i] = get_byte(i * 8, y) + end + + fun work do + var line = 0 + loop + line = atomic.get_and_increment + if line < n then put_line(line, data[line]) else break + end + end +end + +redef class Sys + var n = 0 + var inv_n: Float is noautoinit + var data: Array[Array[Int]] is noautoinit + var crb: Array[Float] is noautoinit + var cib: Array[Float] is noautoinit + var atomic = new AtomicInt(0) + var nb_threads = 8 + # How many time do we iterate before deciding if the number + # is in the mandelbrot set or not + var nb_rounds = 49 +end + +n = if args.is_empty then 200 else args[0].to_i + +sys.crb = new Array[Float].with_capacity(n) +sys.cib = new Array[Float].with_capacity(n) +sys.inv_n = 2.0 / n.to_f +for i in [0..n[ do + sys.cib[i] = i.to_f * inv_n - 1.0 + sys.crb[i] = i.to_f * inv_n - 1.5 +end +sys.data = new Array[Array[Int]].with_capacity(n) +for i in [0..n[ do sys.data[i] = new Array[Int].filled_with(0, (n) / 8) + +# Parallel Approach +var actors = new Array[Worker] +for i in [0..nb_threads[ do + var a = new Worker + actors.add(a) + a.async.work +end + +for a in actors do + a.async.terminate + a.async.wait_termination +end + +var filename = "WRITE".environ +if filename == "" then filename = "out" +var output = new FileWriter.open(filename) +output.write_bytes("P4\n{n} {n}\n".to_bytes) +for i in [0..n[ do + var length = data[i].length + for j in [0..length[ do output.write_byte(data[i][j]) +end +output.close +~~~ + ## Managing actors When you annotate a class with `actor` and create an instance of it with `new`, the actor is not @@ -37,19 +354,60 @@ Actors are not automatically garbage collected, but you have solutions to termin if you need to. For this, you need to use the `async` property of your annotated class : * `async.terminate` sends a shutdown message to the actor telling him to stop, so he'll finish -processing every other messages in his mailbox before terminating properly. Every other messages sent -to this actor after he received the shutdown message won't be processed. + processing every other messages in his mailbox before terminating properly. Every other messages sent + to this actor after he received the shutdown message won't be processed. * `async.terminate_now` sends a shutdown message too, but this time it places it first, so -if the actor is processing one message now, the next one will be the shutdown message, discarding -every messages in its mailbox. + if the actor is processing one message now, the next one will be the shutdown message, discarding + every messages in its mailbox. * `async.wait_termination` wait for the actor to terminate properly. This call is synchronous. * `async.kill`. If you really need this actor to stop, without any regards of what he was doing -or in which state he'll leave the memory, you can with this call. it's synchronous but not really -blocking, since it's direcly canceling the native pthread associated to the actor. + or in which state he'll leave the memory, you can with this call. it's synchronous but not really + blocking, since it's direcly canceling the native pthread associated to the actor. For now, there isn't any mecanism to recreate and actor after it was terminated. Sending messages after terminating it results in unspecified behaviour. +Example from `actors::simple`: + +~~~ +# A very simple example of the actor model +module simple is example + +import actors + + +# A class anotated with `actor` +# It automatically gets the `async` property to make asynchronous calls on it +class A + actor + + # Prints "foo" + fun foo do print "foo" + + # Returns i^2 + fun bar(i : Int): Int do return i * i +end + +# Create a new instance of A +var a = new A + +# Make an asynchronous call +a.async.foo + +# Make a synchronous call +a.foo + +# Make an asynchronous call +# Which return a `Future[Int]` instead of `Int` +var r = a.async.bar(5) + +# Retrieve the value of the future +print r.join + +# Make a Synchronous call +print a.bar(5) +~~~ + ## Waiting for all actors to finish processing Let's imagine you create a whole bunch of actors and make them do things asynchronously from the main thread. @@ -65,8 +423,237 @@ actor, `active_actors` is empty. You can use this property as a mean of synchronisation in some specific cases (for example if you're using actors for fork/join parallelism instead of concurrency). +Example from `actors::fannkuchredux`: + +~~~ +# Example implemented from "The computer Language Benchmarks Game" - Fannkuch-Redux +# http://benchmarksgame.alioth.debian.org/ +# +# Complete description of the fannkuch-redux : +# https://benchmarksgame.alioth.debian.org/u64q/fannkuchredux-description.html#fannkuchredux +module fannkuchredux is example, no_warning("missing-doc") + +import actors + +class FannkuchRedux + actor + + var p: Array[Int] is noautoinit + var pp: Array[Int] is noautoinit + var count: Array[Int] is noautoinit + + fun print_p do + for i in [0..p.length[ do printn p[i] + 1 + print "" + end + + fun first_permutation(idx: Int) do + for i in [0..p.length[ do p[i] = i + + for i in [0..count.length[.reverse_iterator do + var d = idx / fact[i] + count[i] = d + idx = idx % fact[i] + + p.copy_to(0, i+1, pp, 0) + + for j in [0..i] do p[j] = if j + d <= i then pp[j+d] else pp[j+d-i-1] + end + end + + fun next_permutation do + var first = p[1] + p[1] = p[0] + p[0] = first + + var i = 1 + count[i] += 1 + while count[i] > i do + count[i] = 0 + i += 1 + + p[0] = p[1] + var next = p[0] + + for j in [1..i[ do p[j] = p[j+1] + p[i] = first + first = next + + count[i] += 1 + end + end + + fun count_flips: Int do + var flips = 1 + var first = p[0] + if p[first] != 0 then + p.copy_to(0, pp.length, pp, 0) + while pp[first] != 0 do + flips += 1 + var lo = 1 + var hi = first - 1 + while lo < hi do + var t = pp[lo] + pp[lo] = pp[hi] + pp[hi] = t + lo += 1 + hi -= 1 + end + var t = pp[first] + pp[first] = first + first = t + end + end + return flips + end + + fun run_task(task: Int) do + var idx_min = task * chunk_sz + var idx_max = fact[n].min(idx_min + chunk_sz) + + first_permutation(idx_min) + + var maxflips = 1 + var chk_sum = 0 + + for i in [idx_min..idx_max[ do + if p[0] != 0 then + var flips = count_flips + maxflips = maxflips.max(flips) + chk_sum += if i % 2 == 0 then flips else -flips + end + if i + 1 != idx_max then next_permutation + end + + max_flips[task] = maxflips + chk_sums[task] = chk_sum + end + + fun run do + p = new Array[Int].with_capacity(n) + for i in [0..n[ do p.add(0) + pp = new Array[Int].with_capacity(n) + for i in [0..n[ do pp.add(0) + count = new Array[Int].with_capacity(n) + for i in [0..n[ do count.add(0) + + var task = 0 + loop + task = task_id.get_and_increment + if task < n_tasks then run_task(task) else break + end + end +end + +redef class Sys + var n_chunks = 150 + var chunk_sz: Int is noautoinit + var n_tasks: Int is noautoinit + var n: Int is noautoinit + var fact: Array[Int] is noautoinit + var max_flips: Array[Int] is noautoinit + var chk_sums: Array[Int] is noautoinit + var task_id = new AtomicInt(0) + var nb_actors = 8 +end + +fun print_result(n, res, chk: Int) do + print chk.to_s + "\nPfannfuchen(" + n.to_s + ") = " + res.to_s + +end + +n = if args.is_empty then 7 else args[0].to_i + +fact = new Array[Int].with_capacity(n+1) +fact[0] = 1 +for i in [1..n] do fact.add(fact[i-1] * i) + +chunk_sz = (fact[n] + n_chunks - 1) / n_chunks +n_tasks = (fact[n] + chunk_sz - 1) / chunk_sz +max_flips = new Array[Int].with_capacity(n_tasks) +for i in [0..n_tasks[ do max_flips.add(0) +chk_sums = new Array[Int].with_capacity(n_tasks) +for i in [0..n_tasks[ do chk_sums.add(0) + +var actors = new Array[FannkuchRedux].with_capacity(nb_actors) +for i in [0..nb_actors[ do + var a = new FannkuchRedux + actors.add(a) + a.async.run +end + +for i in [0..nb_actors[ do + actors[i].async.terminate + actors[i].async.wait_termination +end + +var res = 0 +for i in max_flips do res = res.max(i) + +var chk = 0 +for i in chk_sums do + chk += i +end + +print_result(n, res, chk) +~~~ ## Examples You can find example of differents small programs implemented with Nit actors in the `examples` directory. For a really simple example, you can check `examples/simple`. + +Example from `actors::agent_simulation`: + +~~~ +# a "Framework" to make Multi-Agent Simulations in Nit +module agent_simulation is example, no_warning("missing-doc") + +import actors + +# Master of the simulation, it initiates the steps and waits for them to terminate +class ClockAgent + actor + + # Number of steps to do in the simulation + var nb_steps: Int + + # List of Agents in the simulation + var agents: Array[Agent] + + # Number of agents that finished their step + var nb_finished = 0 + + fun do_step do + for a in agents do a.async.do_step + nb_steps -= 1 + end + + fun finished_step do + nb_finished += 1 + if nb_finished == agents.length then + nb_finished = 0 + if nb_steps != 0 then async.do_step + end + end +end + +class Agent + actor + + # Doing a step in the simulation + fun do_step do + end + + fun end_step do clock_agent.async.finished_step + +end + +redef class Sys + var clock_agent: ClockAgent is noautoinit,writable +end +~~~ + +## Authors + +This project is maintained by **Romain Chanoir **. diff --git a/lib/ai/README.docdown.md b/lib/ai/README.docdown.md new file mode 100644 index 0000000000..3a976ce224 --- /dev/null +++ b/lib/ai/README.docdown.md @@ -0,0 +1,61 @@ +# `ai` - Simple library for basic artificial intelligence algorithms + +[[toc: ai]] + +This library provides some frameworks for basic algorithms used for artificial intelligence. +It offers simple generic classes to extends. + +Contents: + +* see `backtrack` +* see `search` + +[[features: ai | mentities: ai::ai;ai::backtrack;ai::search]] + +See the `examples` subdirectory for examples: + +* `examples/queens.nit` +* `examples/puzzle.nit` + Example from `ai::puzzle`: + +[[code: ai::puzzle]] + +[[uml: ai | format: svg, mentities: ai::ai;ai::backtrack;ai::search]] + +[[features: ai | mentities: ai::SearchNode;ai::BacktrackNode;ai::BacktrackProblem;ai::BacktrackSolver;ai::SearchProblem;ai::SearchSolver]] + +## [[sign: ai::backtrack]] + +> [[doc: ai::backtrack]] + +### [[sign: ai::BacktrackNode]] + +> [[doc: ai::BacktrackNode]] + +### [[sign: ai::BacktrackProblem]] + +> [[doc: ai::BacktrackProblem]] + +### [[sign: ai::BacktrackSolver]] + +> [[doc: ai::BacktrackSolver]] + +## [[sign: ai::search]] + +> [[doc: ai::search]] + +### [[sign: ai::SearchNode]] + +> [[doc: ai::SearchNode]] + +### [[sign: ai::SearchProblem]] + +> [[doc: ai::SearchProblem]] + +### [[sign: ai::SearchSolver]] + +> [[doc: ai::SearchSolver]] + +## Authors + +This project is maintained by [[ini-maintainer: ai]]. diff --git a/lib/ai/README.md b/lib/ai/README.md index f9997e14cd..54b88c870d 100644 --- a/lib/ai/README.md +++ b/lib/ai/README.md @@ -1,4 +1,14 @@ -Simple library for basic artificial intelligence algorithms +# `ai` - Simple library for basic artificial intelligence algorithms + +* [`backtrack`](#`backtrack`) +* [`BacktrackNode`](#`BacktrackNode`) +* [`BacktrackProblem`](#`BacktrackProblem`) +* [`BacktrackSolver`](#`BacktrackSolver`) +* [`search`](#`search`) +* [`SearchNode`](#`SearchNode`) +* [`SearchProblem`](#`SearchProblem`) +* [`SearchSolver`](#`SearchSolver`) +* [Authors](#Authors) This library provides some frameworks for basic algorithms used for artificial intelligence. It offers simple generic classes to extends. @@ -6,9 +16,498 @@ It offers simple generic classes to extends. Contents: * see `backtrack` + * see `search` +* `ai` - Simple toolkit for artificial intelligence. + +* `backtrack` - Basic framework for active backtrack solver + +* `search` - Basic framework for search problems and solver. + See the `examples` subdirectory for examples: * `examples/queens.nit` * `examples/puzzle.nit` + Example from `ai::puzzle`: + +~~~ +# The N-puzzle problem, modeled naively as a `SearchProblem`. +# +# A square grid, made of tiles represented with a letter, is scrambled. +# A missing tile, the hole, represented with a dot, is used to move them. +# +# The goal is to found a plan, made of the four basic movements up, down, +# left, and right, that move each letter to its correct position: the solved +# grid list letters alphabetically from left to right then top to bottom. +# The hole must be in the last position (bottom-right). +# +# The argument of the program is a initial position, the program then find +# the best plan to solve the puzzle. +# +# ## Example: +# +# The argument "abcd.fgeh" is the grid +# +# ~~~raw +# abc +# d.f +# geh +# ~~~ +# +# The goal is: +# +# ~~~raw +# abc +# def +# gh. +# ~~~ +# +# The shortest plan, in two steps, is to move *up* the tile under the hole (e), +# then to move *left* the tile after the hole (h). +module puzzle is example + +import ai::search + +# The state (`S`) is a square grid, modeled as a one-dimensional array of Tile. +# read from left to right then top to bottom. +# +# An action (`A`) is the relative position of the tile to swap with the hole. +# Therefore, `-1` for left, `1` for right, `-width` for up and `width` for down. +class PuzzleProblem + super SearchProblem[ArrayCmp[Tile], Int] + + # The initial grid. use letters for tiles, and . for the hole. + var initial_grid: String + + # The width of the grid. + # Eg. 3 for a 8-puzzle grid + var width: Int is noinit + + # Construct a state form `initial_grid` + redef fun initial_state do + var g = initial_grid + var len = g.length + var width = len.sqrt.to_i + self.width = width + if width * width != len then + print "Error: {g} has {len} tiles. A square number, like {width*width} is needed" + exit 1 + end + var res = new ArrayCmp[Tile] + for i in [0..g.length[ do + var c = g.chars[i] + if c == ' ' or c == '.' then + var hole = new Tile('.', -1) + self.hole = hole + res.add hole + else if c >= '1' and c <= '9' then + var t = new Tile(c, '1'.distance(c)) + res.add t + else if c >= 'a' and c <= 'z' then + var t = new Tile(c, 'a'.distance(c)) + res.add t + else if c >= 'A' and c <= 'Z' then + var t = new Tile(c, 'A'.distance(c)) + res.add t + else + print "Error: illegal tile {c} in {g}" + exit 1 + end + end + return res + end + + # Get the four available movements, or 3 on a edge, or 2 in a corner. + redef fun actions(state) + do + var h = get_hole(state) + var x = h % width + var y = h / width + var res = new Array[Int] + if x >= 1 then res.add(-1) + if x < width-1 then res.add(1) + if y >= 1 then res.add(-width) + if y < width-1 then res.add(width) + return res + end + + # Return the state where the tile at hole+action has moved + redef fun apply_action(state, action) + do + # Copy the state + var res = new ArrayCmp[Tile].with_capacity(state.length) + res.add_all(state) + + # Get the hole and the tile next to it + var h = get_hole(res) + var t = h + action + + # Move (by swapping the tile and the hole) + res[h] = res[t] + res[t] = hole + + return res + end + + # The special empty tile for fast retrieval. + var hole: Tile is noinit + + # What is the position of the hole? + fun get_hole(state: Array[Tile]): Int + do + return state.index_of(hole) + end + + # Each tile is at its correct position + redef fun is_goal(state) + do + for i in [0..state.length[ do + var t = state[i] + if t != hole and t.goal_idx != i then return false + end + return true + end + + # The sum of the Manhattan distances of each tile to its goal + # Not the best heuristic but the simplest to implement among the good ones. + redef fun heuristic(state) + do + var p = 0 + var i = -1 + for t in state do + i += 1 + + # The hole does not count + if t == hole then continue + + var dx = (i % width - t.goal_idx % width).abs + var dy = (i / width - t.goal_idx / width).abs + + # Add Manhattan distance + p += dx + dy + end + return p.to_f + end + + # Print the grid + fun print_state(state: Array[Tile]) + do + for i in [0..state.length[ do + var t = state[i] + if t == hole then + printn "." + else + printn t.symbol + end + if (i+1) % width == 0 then print "" + end + end + + # Print the plan with words. + fun print_plan(plan: Sequence[Int]) + do + var s = new Array[String] + for i in plan do + if i == -1 then + s.add "right(>)" + else if i == 1 then + s.add "left(<)" + else if i == -width then + s.add "down(v)" + else if i == width then + s.add "up(^)" + else + abort + end + end + print "Solution in {plan.length} moves: {s.join(" ")}" + end + + redef fun make_memory do + var res = super + res.add new RBTreeMap[ArrayCmp[Tile], SearchNode[ArrayCmp[Tile], Int]] + res.add new BinTreeMap[ArrayCmp[Tile], SearchNode[ArrayCmp[Tile], Int]] + return res + end +end + +# A movable tile +# A simple class to encapsulate the symbol and the goal position. +class Tile + super Comparable + redef type OTHER: Tile is fixed + + # The symbol written on the tile + var symbol: Char + + # The expected position in the grid + var goal_idx: Int + + redef fun to_s do return symbol.to_s + redef fun ==(o) do return o isa Tile and goal_idx == o.goal_idx + redef fun <(o) do return goal_idx < o.goal_idx + redef fun <=>(o) do return goal_idx <=> o.goal_idx +end + +var configs = false + +if not args.is_empty then + if args.first == "--configs" then + configs = true + args.shift + end +end + +if args.is_empty then + print """ +Usage: puzzle [--configs] initial... + +--configs: search and time solutions with various configurations of solvers. +initial: an initial configuration (letters for the tiles, and dot for the hole). eg: + + 8-puzzle: + + goal (0): abcdefgh. + easy (4): abce.fdgh + medium (10): eabf.cdgh + hard (20): feacbh.dg + harder (31): hfgbedc.a + + 15-puzzle: + goal (0): abcdefghijklmno. + easy (30): bacdefghijlkmno. + medium (40): fg.jacoheldnibmk + hard (55): kleg.mondcafjhbi + harder (61): lomgkcend.afjhbi + + 24-puzzle: + goal (0): abcdefghijklmnopqrstuvwx. + easy (55): giabcjekmdhrtflqsownpuv.x + medium (75): giabcjekmdrtwulhs.vnqofpx + hard (79): giabcjekmdrtw.uhsvnlqofpx + harder (80): giabcjekmdrt.wuhsvnlqofpx +""" + exit 0 +end + + +for arg in args do + var pb = new PuzzleProblem(arg) + print "Initial: {arg}" + pb.print_state(pb.initial_state) + + if configs then + pb.run_configs(1000000) + continue + end + + var s = pb.astar + s.memorize = true + var r = s.run + if r == null then + print "No solution." + break + end + + print "Solved, after looking at {r.steps} positions" + pb.print_plan(r.plan) +end +~~~ + +![Diagram for `ai`](uml-ai.svg) + +* `BacktrackNode` - A node in the backtrack-zipper visited by a `BacktrackSolver`. +* `BacktrackProblem` - Abstract backtrack problem of states (`S`) and actions (`A`). +* `BacktrackSolver` - A running solver for a given problem, that can be configured and controlled. +* `SearchNode` - A node in the search-tree visited by a `SearchSolver`. +* `SearchProblem` - Abstract search problem over immutable states (`S`) and actions (`A`). +* `SearchSolver` - A running solver for a given problem, to configure and control. + +## `backtrack` + +> This module provides a simple abstract class `BacktrackProblem[S,A]` to be specialized for a specific problem. + +The concrete class `BacktrackSolver` is used to configure, query, and run a solver for a given problem. + +For an example, see the `queens.nit` program in the `examples` subdirectory. + +### `BacktrackNode` + +> The solver visits the virtual search tree with a zipper. + +A node is the zipper (this class) is associated to: + +* a state of the problem (indirectly), +* the actions not yet explored from the state (see `totry`) +* the action that yields to the state (see `action`), used to backtrack. +* and the parent node in the zipper (see `parent`). + +There is no direct link between a node and a state; it is unneeded +since the same state is used, and mutated, during the whole execution of the solver. + +This class is exposed to allow queries on the solution provided by the solver. + +### `BacktrackProblem` + +> This class serves to model search problems using a backtracking approach. +> A state, `S`, is a point in the search problem and fully model a given state of the world. +> An action, `A`, is an available mean of transition between two states. +> While there is a potential large number of distinct states and actions, there should be only +> a small number of possible actions from a specific state (thus, a small, or at least finite, branching factor). + +The point this class is that the state is a mutable object, the roles of the actions is to modify +the state. + +This abstract class is generic and made to work with any kind of states and actions. +Therefore, specific subclasses must be developed to implements the required services: + +* `initial_state` +* `actions` +* `apply_action` +* `backtrack` +* `is_goal` + +# Basic search + +The method `solve` returns a new solver for a backtrack search. + +### `BacktrackSolver` + +> # Basic run and results. + +1. Instantiate it with the method `solve` from `BacktrackProblem`. +2. Apply the method `run`, that will search and return a solution. +3. Retrieve information from the solution. + +~~~~nitish +var p: BacktrackProblem = new MyProblem +var solver = p.solve +var res = solver.run +if res != null then + print "Found solution in {res.depth} actions: {res.plan.join(", ")}" + print "The state of the solution is: {solver.state}" +end +~~~~ + +# Step-by-step runs and multiple runs + +The `run_steps` method (see also `steps`, and `steps_limit`) can be used to run only a maximum number of steps. +Thus, this method can be used as a *co-routine* and be run periodically for a small amount of time. + +`run` and `run_steps` return the next solution. +A subsequent call to `run` returns the following solution and so on. + +When there is no more solutions available, `null` is returned and `is_running` become false. + +Between run, the state of the current search can be read. + +# Search-trees + +Internally, solvers use a zipper on the virtual search-tree where nodes are elements in the apply/backtrack graph. +See the class `BacktrackNode` for details + +The `run` and `node` methods return a `BacktrackNode` that can be used to retrieve a lot of useful information, +like the full `path` or the `plan`. +If only the solved state is required, the `state` method from the solver gives it. + +## `search` + +> The module provides a simple abstract class `SearchProblem[S,A]` to be specialized for a specific problem. + +The concrete class `SearchSolver` is used to configure, query, and run a solver for a given problem. + +For an example, see the `puzzle.nit` program in the `examples` subdirectory. + +### `SearchNode` + +> The root node is labeled by the initial state of the problem. + +This class is exposed to allow queries on the solution provided by the solver. + +### `SearchProblem` + +> This class serves to model problems of planing and path-finding. +> A state, `S`, is a point in the search problem and fully models a given state of the world. +> An action, `A`, is an available mean of transition between two states. + +This abstract class is generic made to work with any kind of states and actions. +Therefore, specific subclasses must be developed to implement the required services: + +* `initial_state` +* `actions` +* `apply_action` +* `is_goal` + +Note that the implemented methods should not temper with the parameters since it is expected +that they remain unmodified. + +# Basic search + +These tree method are enough to trigger a basic search. + +The methods `breadth_first` and `depth_first` return pre-configured solvers for, respectively, +a breadth-first search, a depth-first search. + +# Advanced search + +The `cost` method can be implemented to represent the cost of a single transition. +By default, the cost is 1. + +The `heuristic` method can be implemented to represent a lower-bound estimation of the remaining +cost to reach a goal state. +By default, the heuristic is 0. + +If one of these (or both) methods are implemented, the `astar` method will return a pre-configured +solver for a A* search. + +More configuration and optimization on the search can to be done in the `SearchSolver` class. + +### `SearchSolver` + +> For a given problem, a lot of variation of search algorithms can be made. +> Thus this class permit the user to control the parameters of the search algorithm. + +Note that this class is not meant to be specialized, and usually not instantiated directly. + +# Basic run and result. + +1. Instantiate it with the method `breadth_first`, `depth_first`, or `astar` from `SearchProblem`. +2. Apply the method `run`, that will search and return a solution. +3. Retrieve information from the solution. + +~~~~nitish +var p: SearchProblem = new MyProblem +var res = p.astar.run +if res != null then print "Found plan with {res.depth} actions, that cost {res.cost}: {res.plan.join(", ")}" +~~~~ + +# Step-by-step runs and multiple runs + +The `run_steps` method (see also `steps`, and `steps_limit`) can be used to run only a maximum number of steps. +This method can be used as a *co-routine* and run them periodically for a small amount of time. + +`run` and `run_steps` return the next solution. +A subsequent call to `run` returns the following solution and so on. + +When there is no more solutions available, `is_running` become false. + +# Search-trees + +Internally, solvers use a search-tree where nodes are labeled with states, and edges are labeled with actions. +See `SearchNode` for details. + +The `run` method return a `SearchNode` that can be used to retrieve a lot of useful information, +like the full `path` or the `plan`. + +# Configuration + +The solver provide some *knobs* to control how the search-tree is visited. + +* `memorize` (see also `memorize_late`) +* `do_revisit` (see also `revisits`) +* `depth_limit` (see also `iterative_deepening` and `depth_limit_reached`) + +## Authors + +This project is maintained by **Jean Privat **. diff --git a/lib/android/README.docdown.md b/lib/android/README.docdown.md new file mode 100644 index 0000000000..9b3c2185a6 --- /dev/null +++ b/lib/android/README.docdown.md @@ -0,0 +1,264 @@ +# `android` - Android platform support and APIs + +[[toc: android]] + +> [[doc: android::android]] + +## Getting Started + +These instructions will get you a copy of the project up and running on your local machine. + +### Dependencies + +This project requires the following packages: + +[[parents: android]] + +### Run `nit_activity` + +[[uml: android | format: svg, mentities: android::portrait;android::landscape;android::android;android::at_boot;android::shared_preferences;android::intent_api12;android::intent_api17;android::intent_api19;android::intent_api14;android::platform;android::intent_api18;android::activities;android::intent_api11;android::vibration;android::toast;android::cardboard;android::native_notification;android::data_store;android::aware;android::intent_api15;android::intent;android::key_event;android::wifi;android::game;android::shared_preferences_api11;android::dalvik;android::assets;android::service;android::log;android::intent_api16;android::load_image;android::http_request;android::native_app_glue;android::nit_activity;android::notification;android::gamepad;android::ui;android::native_ui;android::input_events;android::sensors;android::assets_and_resources;android::shared_preferences_api10;android::bundle;android::audio;android::intent_api10]] + +Compile `nit_activity` with the following command: + +[[main-compile: android::nit_activity]] + +Then run it with: + +[[main-run: android::nit_activity]] + +## Compilation for Android + +The compiler generates an APK file as the output when the [[android | text: `android`]] +module is imported by the compilation target. The path to the generated +file can be specified using the `-o` and `--dir` options. + +## Host system configuration + +To compile Android apps from a 64 bits GNU/Linux host you can reuse an existing Android Studio +installation or make a clean install with command line tools only. + +Note that this guide supports only 64 bits GNU/Linux hosts with support for a Java 8 JDK, +it may be possible to support other platforms with some tweaks. + +1. Install the required SDK packages using one of these two methods: + + a. Using Android Studio, open `Tools > Android > SDK Manager`, in the SDK Tools tab, + install "Android SDK Build-Tools", CMake and NDK. + + b. From the command line, run this script for a quick setup without Android Studio. + You will probably need to tweak it to you system or update the download URL + to the latest SDK tools from https://developer.android.com/studio/index.html#command-tools + + ~~~bash + # Fetch and extract SDK tools + mkdir -p ~/Android/Sdk + cd ~/Android/Sdk + wget https://dl.google.com/android/repository/sdk-tools-linux-3859397.zip + unzip sdk-tools-linux-3859397.zip + + # Update tools + tools/bin/sdkmanager --update + + # Accept the licenses + tools/bin/sdkmanager --licenses + + # Install the basic build tools + tools/bin/sdkmanager "build-tools;27.0.0" ndk-bundle + ~~~ + +2. Set the environment variable ANDROID_HOME to the SDK installation directory, usually `~/Android/Sdk/`. + Use the following command to setup the variable for bash. + + ~~~bash + echo "export ANDROID_HOME=~/Android/Sdk/" >> ~/.bashrc + ~~~ + +3. Install Java 8 JDK, on Debian/Ubuntu systems you can use the following command: + + ~~~bash + sudo apt install openjdk-8-jdk + ~~~ + +## Configure the Android application + +The _app.nit_ framework and this project offers some [[android>service> | text: services]] to +customize the generated Android application. + +### Annotations + +* All _app.nit_ annotations are applied to Android projects: + `app_name`, `app_namespace` and `app_version`. + + See: `../app/README.md` + +* Custom information can be added to the Android manifest file + using the annotations `android_manifest`, `android_manifest_application` + and `android_manifest_activity`. + + Example usage to specify an extra permission: + + ~~~ + android_manifest """""" + ~~~ + +* The API version target can be specified with `android_api_min`, + `android_api_max` and `android_api_target`. These take a single + integer as argument. They are applied in the Android manifest as + `minSdkVesion`, `targetSdkVersion` and `maxSdkVersion`. + + See http://developer.android.com/guide/topics/manifest/uses-sdk-element.html + +* The annotation `android_activity` defines a Java class used as an + entry point to your application. As of now, this annotation should + only be used by low-level implementations of Nit on Android. + Its usefulness will be extended in the future to customize user applications. + +### Android implementation + +There is two core implementation for Nit apps on Android. +`android::nit_activity` is used by apps with standard windows and native [[android>ui> | text: UI]] controls. +`android::game` is used by, well, games and the game frameworks `mnit` and `gamnit`. + +Clients don't have to select the core implementation, it is imported by other relevant modules. +For example, a module importing `app::ui` and [[android::android | text: `android`]] will trigger the importation of [[android::nit_activity | text: `android::nit_activity`]]. + +[[features: android | mentities: android::portrait;android::landscape;android::android;android::at_boot;android::shared_preferences;android::intent_api12;android::intent_api17;android::intent_api19;android::intent_api14;android::platform;android::intent_api18;android::activities;android::intent_api11;android::vibration;android::toast;android::cardboard;android::native_notification;android::data_store;android::aware;android::intent_api15;android::intent;android::key_event;android::wifi;android::game;android::shared_preferences_api11;android::dalvik;android::assets;android::service;android::log;android::intent_api16;android::load_image;android::http_request;android::native_app_glue;android::nit_activity;android::notification;android::gamepad;android::ui;android::native_ui;android::input_events;android::sensors;android::assets_and_resources;android::shared_preferences_api10;android::bundle;android::audio;android::intent_api10]] + +### Lock app orientation + +Importing `android::landscape` or `android::portrait` locks the generated +application in the specified orientation. This can be useful for games and +other multimedia applications. + +#### [[sign: android::landscape]] + +> [[doc: android::landscape]] + +### Resources and application icon + +Resources specific to the Android platform should be placed in an `android/` folder at the root of the project. +The folder should adopt the structure of a normal Android project, e.g., a custom XML resource file can be placed +at `android/res/values/color.xml` to be compiled with the Android application. + +The application icon should also be placed in the `android/` folder. +Place the classic bitmap version at `android/res/mipmap-hdpi/ic_launcher.png` (and others), +and the adaptive version at `android/res/mipmap-anydpi-v26/ic_launcher.xml`. +The Nit compiler detects these files and uses them as the application icon. + +Additional `android/` folders may be placed next to more specific Nit modules to change the Android resources +for application variants. The more specific resources will have priority over the project level `android/` files. + +#### [[sign: android::assets_and_resources]] + +> [[doc: android::assets_and_resources]] + +#### [[sign: android::assets]] + +> [[doc: android::assets]] + +## Compilation modes + +There are two compilation modes for the Android platform, debug and release. +Theses modes are also applied to the generated Android projects. +The compilation mode is specified as an argument to `nitc`, only +`--release` can be specified as debug is the default behavior. + +### Debug mode + +Debug mode enables compiling to an APK file without handling signing keys +and their password. The APK file can be installed to a local device with +USB debugging enabled, but it cannot be published on the Play Store. + +By default, `nitc` will compile Android applications in debug mode. + +Example from `android::ui_test`: + +[[code: android::ui_test]] + +### Release mode + +Building in release mode will use your private key to sign the +APK file, it can then be published on the Play Store. + +1. Have a keystore with a valid key to sign your APK file. + + To create a new keystore, avoid using the default values of `jarsigner` + as they change between versions of the Java SDK. You should instead use a + command similar to the following, replacing `KEYSTORE_PATH` and `KEY_ALIAS` + with the desired values. + + ~~~bash + keytool -genkey -keystore KEYSTORE_PATH -alias KEY_ALIAS -sigalg MD5withRSA -keyalg RSA -keysize 1024 -validity 10000 + ~~~ + +2. Set the environment variables used by `nitc`: `KEYSTORE`, `KEY_ALIAS` and + optionally `TSA_SERVER`. These settings can be set in a startup script such as + `~/.bashrc` or in a local Makefile. + + You can use the following commands by replacing the right-hand values + to your own configuration. + + ~~~bash + export KEYSTORE=keystore_path + export KEY_ALIAS=key_alias + export TSA_SERVER=timestamp_authority_server_url # Optional + ~~~ + +3. Call `nitc` with the `--release` options. You will be prompted for the + required passwords as needed by `jarsigner`. + +## [[sign: android::android]] + +> [[doc: android::android]] + +## [[sign: android::shared_preferences]] + +> [[doc: android::shared_preferences]] + +## [[sign: android::vibration]] + +> [[doc: android::vibration]] + +## [[sign: android::cardboard]] + +> [[doc: android::cardboard]] + +## [[sign: android::data_store]] + +> [[doc: android::data_store]] + +## [[sign: android::aware]] + +> [[doc: android::aware]] + +## [[sign: android::intent]] + +> [[doc: android::intent]] + +## [[sign: android::wifi]] + +> [[doc: android::wifi]] + +## [[sign: android::native_app_glue]] + +> [[doc: android::native_app_glue]] + +## [[sign: android::nit_activity]] + +> [[doc: android::nit_activity]] + +## [[sign: android::notification]] + +> [[doc: android::notification]] + +## [[sign: android::sensors]] + +> [[doc: android::sensors]] + +## [[sign: android::audio]] + +> [[doc: android::audio]] + +## Authors + +This project is maintained by [[ini-maintainer: android]]. diff --git a/lib/android/README.md b/lib/android/README.md index ffd961cc5d..a0c07c4d0b 100644 --- a/lib/android/README.md +++ b/lib/android/README.md @@ -1,12 +1,78 @@ -Android platform support and APIs - -# Compilation for Android - -The compiler generates an APK file as the output when the `android` +# `android` - Android platform support and APIs + +* [Getting Started](#Getting-Started) +* [Dependencies](#Dependencies) +* [Run `nit_activity`](#Run-`nit_activity`) +* [Compilation for Android](#Compilation-for-Android) +* [Host system configuration](#Host-system-configuration) +* [Configure the Android application](#Configure-the-Android-application) +* [Annotations](#Annotations) +* [Android implementation](#Android-implementation) +* [Lock app orientation](#Lock-app-orientation) +* [`landscape`](#`landscape`) +* [Resources and application icon](#Resources-and-application-icon) +* [`assets_and_resources`](#`assets_and_resources`) +* [`assets`](#`assets`) +* [Compilation modes](#Compilation-modes) +* [Debug mode](#Debug-mode) +* [Release mode](#Release-mode) +* [`android`](#`android`) +* [`shared_preferences`](#`shared_preferences`) +* [`vibration`](#`vibration`) +* [`cardboard`](#`cardboard`) +* [`data_store`](#`data_store`) +* [`aware`](#`aware`) +* [`intent`](#`intent`) +* [`wifi`](#`wifi`) +* [`native_app_glue`](#`native_app_glue`) +* [`nit_activity`](#`nit_activity`) +* [`notification`](#`notification`) +* [`sensors`](#`sensors`) +* [`audio`](#`audio`) +* [Authors](#Authors) + +> This module provides basic logging facilities, advanced logging can be +> achieved by importing `android::log`. + +## Getting Started + +These instructions will get you a copy of the project up and running on your local machine. + +### Dependencies + +This project requires the following packages: + +* `app` - _app.nit_, a framework for portable applications +* `c` - Structures and services for compatibility with the C language +* `core` - Nit common library of core classes and methods +* `gamnit` - Portable game and multimedia framework for Nit +* `java` - Supporting services for the FFI with Java and to access Java libraries +* `json` - read and write JSON formatted text +* `mnit` + +### Run `nit_activity` + +![Diagram for `android`](uml-android-3.svg) + +Compile `nit_activity` with the following command: + +~~~bash +nitc ./nit_activity.nit +~~~ + +Then run it with: + +~~~bash +./nit_activity +~~~ + +## Compilation for Android + +The compiler generates an APK file as the output when the [`android`](android) module is imported by the compilation target. The path to the generated file can be specified using the `-o` and `--dir` options. -# Host system configuration +## Host system configuration To compile Android apps from a 64 bits GNU/Linux host you can reuse an existing Android Studio installation or make a clean install with command line tools only. @@ -14,95 +80,145 @@ installation or make a clean install with command line tools only. Note that this guide supports only 64 bits GNU/Linux hosts with support for a Java 8 JDK, it may be possible to support other platforms with some tweaks. -1. Install the required SDK packages using one of these two methods: +1. Install the required SDK packages using one of these two methods: - a. Using Android Studio, open `Tools > Android > SDK Manager`, in the SDK Tools tab, - install "Android SDK Build-Tools", CMake and NDK. + a. Using Android Studio, open `Tools > Android > SDK Manager`, in the SDK Tools tab, + install "Android SDK Build-Tools", CMake and NDK. - b. From the command line, run this script for a quick setup without Android Studio. - You will probably need to tweak it to you system or update the download URL - to the latest SDK tools from https://developer.android.com/studio/index.html#command-tools + b. From the command line, run this script for a quick setup without Android Studio. + You will probably need to tweak it to you system or update the download URL + to the latest SDK tools from https://developer.android.com/studio/index.html#command-tools - ~~~ - # Fetch and extract SDK tools - mkdir -p ~/Android/Sdk - cd ~/Android/Sdk - wget https://dl.google.com/android/repository/sdk-tools-linux-3859397.zip - unzip sdk-tools-linux-3859397.zip + ~~~bash + # Fetch and extract SDK tools + mkdir -p ~/Android/Sdk + cd ~/Android/Sdk + wget https://dl.google.com/android/repository/sdk-tools-linux-3859397.zip + unzip sdk-tools-linux-3859397.zip - # Update tools - tools/bin/sdkmanager --update + # Update tools + tools/bin/sdkmanager --update - # Accept the licenses - tools/bin/sdkmanager --licenses + # Accept the licenses + tools/bin/sdkmanager --licenses - # Install the basic build tools - tools/bin/sdkmanager "build-tools;27.0.0" ndk-bundle - ~~~ + # Install the basic build tools + tools/bin/sdkmanager "build-tools;27.0.0" ndk-bundle + ~~~ -3. Set the environment variable ANDROID_HOME to the SDK installation directory, usually `~/Android/Sdk/`. - Use the following command to setup the variable for bash. +2. Set the environment variable ANDROID_HOME to the SDK installation directory, usually `~/Android/Sdk/`. + Use the following command to setup the variable for bash. - ~~~ - echo "export ANDROID_HOME=~/Android/Sdk/" >> ~/.bashrc - ~~~ + ~~~bash + echo "export ANDROID_HOME=~/Android/Sdk/" >> ~/.bashrc + ~~~ -4. Install Java 8 JDK, on Debian/Ubuntu systems you can use the following command: +3. Install Java 8 JDK, on Debian/Ubuntu systems you can use the following command: - ~~~ - sudo apt install openjdk-8-jdk - ~~~ + ~~~bash + sudo apt install openjdk-8-jdk + ~~~ -# Configure the Android application +## Configure the Android application -The _app.nit_ framework and this project offers some services to +The _app.nit_ framework and this project offers some [services](android>service>) to customize the generated Android application. -## Annotations +### Annotations * All _app.nit_ annotations are applied to Android projects: `app_name`, `app_namespace` and `app_version`. - See: `../app/README.md` + See: `../app/README.md` * Custom information can be added to the Android manifest file -using the annotations `android_manifest`, `android_manifest_application` -and `android_manifest_activity`. + using the annotations `android_manifest`, `android_manifest_application` + and `android_manifest_activity`. - Example usage to specify an extra permission: + Example usage to specify an extra permission: - ~~~ - android_manifest """""" - ~~~ + ~~~ + android_manifest """""" + ~~~ * The API version target can be specified with `android_api_min`, -`android_api_max` and `android_api_target`. These take a single -integer as argument. They are applied in the Android manifest as -`minSdkVesion`, `targetSdkVersion` and `maxSdkVersion`. + `android_api_max` and `android_api_target`. These take a single + integer as argument. They are applied in the Android manifest as + `minSdkVesion`, `targetSdkVersion` and `maxSdkVersion`. - See http://developer.android.com/guide/topics/manifest/uses-sdk-element.html + See http://developer.android.com/guide/topics/manifest/uses-sdk-element.html * The annotation `android_activity` defines a Java class used as an - entry point to your application. As of now, this annotation should + entry point to your application. As of now, this annotation should only be used by low-level implementations of Nit on Android. Its usefulness will be extended in the future to customize user applications. -## Android implementation +### Android implementation There is two core implementation for Nit apps on Android. -`android::nit_activity` is used by apps with standard windows and native UI controls. +`android::nit_activity` is used by apps with standard windows and native [UI](android>ui>) controls. `android::game` is used by, well, games and the game frameworks `mnit` and `gamnit`. Clients don't have to select the core implementation, it is imported by other relevant modules. -For example, a module importing `app::ui` and `android` will trigger the importation of `android::nit_activity`. - -## Lock app orientation +For example, a module importing `app::ui` and [`android`](android::android) will trigger the importation of [`android::nit_activity`](android::nit_activity). + +* `activities` - Android Activities wrapper +* `android` - Android services and implementation of app.nit +* `assets` - Implementation of `app::assets` +* `assets_and_resources` - Android Assets and Resources Management +* `at_boot` - Import this module to launch `Service` at device boot +* `audio` - Android audio services, wraps a part of android audio API +* `aware` - Android compatibility module +* `bundle` - A mapping class of `String` to various value types used by the +* `cardboard` - Services from the Google Cardboard SDK for virtual reality on Android +* `dalvik` - Java related services specific to Android and its Dalvik VM +* `data_store` - Implements `app::data_store` using `shared_preferences` +* `game` - Android services and implementation of app.nit for gamnit and mnit +* `gamepad` - Support for gamepad events (over Bluetooth or USB) +* `http_request` - Android implementation of `app:http_request` +* `input_events` - Pointer and hardware key events +* `intent` - Services allowing to launch activities and start/stop services using +* `intent_api10` - Services allowing to launch activities and start/stop services using +* `intent_api11` - Refines intent module to add API 11 services +* `intent_api12` - Refines intent module to add API 12 services +* `intent_api14` - Refines intent module to add API 14 services +* `intent_api15` - Refines intent module to add API 15 services +* `intent_api16` - Refines intent module to add API 16 services +* `intent_api17` - Refines intent module to add API 17 services +* `intent_api18` - Refines intent module to add API 18 services +* `intent_api19` - Refines intent module to add API 19 services +* `key_event` +* `landscape` - Lock the application in the landscape orientation +* `load_image` - Low-level services to load pixel data from the assets +* `log` - Advanced Android logging services +* `native_app_glue` - Wrapper of the Android native_app_glue framework to implement app.nit +* `native_notification` - Native Java classes for notifications +* `native_ui` - Native services from the `android.view` and `android.widget` namespaces +* `nit_activity` - Core implementation of `app.nit` on Android using a custom Java entry point +* `notification` - Services to show notification in the Android status bar +* `platform` - Triggers compilation for the android platform +* `portrait` - Config to set the portrait orientation +* `sensors` - Access Android sensors +* `service` - Android service support for _app.nit_ centered around the class `Service` +* `shared_preferences` - Services allowing to save and load datas to internal android device +* `shared_preferences_api10` - Services to save/load data using `android.content.SharedPreferences` for the android platform +* `shared_preferences_api11` - Refines shared_preferences module to add API 11 services +* `toast` - Services to display a _toast_, a small popup on Android +* `ui` - Views and services to use the Android native user interface +* `vibration` - Vibration services for Android +* `wifi` - Simple wrapper of the Android WiFi services + +### Lock app orientation Importing `android::landscape` or `android::portrait` locks the generated application in the specified orientation. This can be useful for games and other multimedia applications. -## Resources and application icon +#### `landscape` + +> Adds `android:screenOrientation=="sensorLandscape"` to the manifest. + +### Resources and application icon Resources specific to the Android platform should be placed in an `android/` folder at the root of the project. The folder should adopt the structure of a normal Android project, e.g., a custom XML resource file can be placed @@ -116,14 +232,28 @@ The Nit compiler detects these files and uses them as the application icon. Additional `android/` folders may be placed next to more specific Nit modules to change the Android resources for application variants. The more specific resources will have priority over the project level `android/` files. -# Compilation modes +#### `assets_and_resources` + +> Use the ResourceManager to retrieve resources from the `res` folder of your app +> Use the AssetManager to retrieve resources files from the `assets` folder of your app +> both are available from `App` +> If you write your own resources in your NIT project part of the application, +> you are obliged to set a string resource with the name "app_name" or it will be +> impossible for you to compile the apk correctly + +#### `assets` + +> This module is a client of `assets_and_resources` as the latter +> covers the services of Android. + +## Compilation modes There are two compilation modes for the Android platform, debug and release. Theses modes are also applied to the generated Android projects. The compilation mode is specified as an argument to `nitc`, only `--release` can be specified as debug is the default behavior. -## Debug mode +### Debug mode Debug mode enables compiling to an APK file without handling signing keys and their password. The APK file can be installed to a local device with @@ -131,34 +261,297 @@ USB debugging enabled, but it cannot be published on the Play Store. By default, `nitc` will compile Android applications in debug mode. -## Release mode +Example from `android::ui_test`: + +~~~ +# Test for app.nit's UI services +module ui_test is + example + app_name "app.nit UI test" + app_version(0, 1, git_revision) + app_namespace "org.nitlanguage.ui_test" + android_manifest_activity """android:theme="@android:style/Theme.Light"""" + android_api_target 15 +end + +import android::ui +import android::toast +import android::notification + +redef class App + redef fun on_create + do + self.window = new Window + super + end +end + +redef class Window + + private var layout = new VerticalLayout(parent=self) + + private var but_notif = new Button(parent=layout, text="Show Notification") + private var but_toast = new Button(parent=layout, text="Show Toast") + + private var notif: nullable Notification = null + + init + do + but_notif.observers.add self + but_toast.observers.add self + end + + # Action when pressing `but_notif` + fun act_notif + do + var notif = self.notif + if notif == null then + notif = new Notification("From app.nit", "Some content...") + notif.ticker = "Ticker text..." + notif.show + self.notif = notif + else + notif.cancel + self.notif = null + end + end + + # Action when pressing `but_toast` + fun act_toast + do + app.toast("Sample toast from app.nit at {get_time}", false) + end + + redef fun on_event(event) + do + print "on_event {event}" + if event isa ButtonPressEvent then + var sender = event.sender + if sender == but_notif then + act_notif + else if sender == but_toast then + act_toast + end + end + end +end +~~~ + +### Release mode Building in release mode will use your private key to sign the APK file, it can then be published on the Play Store. 1. Have a keystore with a valid key to sign your APK file. - To create a new keystore, avoid using the default values of `jarsigner` -as they change between versions of the Java SDK. You should instead use a -command similar to the following, replacing `KEYSTORE_PATH` and `KEY_ALIAS` -with the desired values. + To create a new keystore, avoid using the default values of `jarsigner` + as they change between versions of the Java SDK. You should instead use a + command similar to the following, replacing `KEYSTORE_PATH` and `KEY_ALIAS` + with the desired values. - ~~~ - keytool -genkey -keystore KEYSTORE_PATH -alias KEY_ALIAS -sigalg MD5withRSA -keyalg RSA -keysize 1024 -validity 10000 - ~~~ + ~~~bash + keytool -genkey -keystore KEYSTORE_PATH -alias KEY_ALIAS -sigalg MD5withRSA -keyalg RSA -keysize 1024 -validity 10000 + ~~~ 2. Set the environment variables used by `nitc`: `KEYSTORE`, `KEY_ALIAS` and -optionally `TSA_SERVER`. These settings can be set in a startup script such as -`~/.bashrc` or in a local Makefile. + optionally `TSA_SERVER`. These settings can be set in a startup script such as + `~/.bashrc` or in a local Makefile. - You can use the following commands by replacing the right-hand values -to your own configuration. + You can use the following commands by replacing the right-hand values + to your own configuration. - ~~~ - export KEYSTORE=keystore_path - export KEY_ALIAS=key_alias - export TSA_SERVER=timestamp_authority_server_url # Optional - ~~~ + ~~~bash + export KEYSTORE=keystore_path + export KEY_ALIAS=key_alias + export TSA_SERVER=timestamp_authority_server_url # Optional + ~~~ 3. Call `nitc` with the `--release` options. You will be prompted for the -required passwords as needed by `jarsigner`. + required passwords as needed by `jarsigner`. + +## `android` + +> This module provides basic logging facilities, advanced logging can be +> achieved by importing `android::log`. + +## `shared_preferences` + +> By default, the API 10 is imported. You can import API 11 to have +> access to this platform new features. + +## `vibration` + +> Importing this module will trigger the use of the VIBRATE permission + +## `cardboard` + +> Projects using this module should keep the `cardboard.jar` archive in the +> `libs` folder at the root of the project. + +External resources: + +* Download `cardboard.jar` from + https://raw.githubusercontent.com/googlesamples/cardboard-java/master/CardboardSample/libs/cardboard.jar +* Read about Cardboard at + https://developers.google.com/cardboard/ +* Find the Cardboard SDK documentation at + https://developers.google.com/cardboard/android/latest/reference/com/google/vrtoolkit/cardboard/package-summary + +## `data_store` + +> We use the shared preferences named "data_store" to store the data. + +## `aware` + +> Defines all Android related annotations, including `ldflags@android`. + +## `intent` + +> By default, API 10 is imported. Import more recent API to suit your needs. + +There's two ways of defining which activity/service to be launched : + +* Provide an explicit class to be launched with `set_class_name` +* Provide a description of the activity to perform and let the system determine the best application to run + +To provide a description, you need to assign values using one or more of these +methods : + +* `action=` +* `data=` +* `add_category` +* `mime_type=` + +The Intent class wraps most of the `android.content.Intent` constants and is +designed to be used with this syntax : + +* Action : `intent_action.main.to_s` +* Category : `intent_category.home.to_s` +* Extra : `intent_extra.phone_number.to_s` +* Flag : `intent_flag.activity_brought_to_front` + +For further details about these constants meaning, refer to the official +android Intent documentation : + +## `wifi` + +> Refer to the official Android documentation for the details on these services. + +## `native_app_glue` + +> The framework provides 3 different structures for a single C application +> under Android. We use all 3 structures in this API to implement app.nit +> on Android. Each structure is wrapped in a Nit extern class: + +* `NativeAppGlue` is the lowest level class, it is implemented by the C + structure `android_app`. It offers features on the main Android thread + (not on the same thread as Nit). For this reason, prefer to use + `NdkNativeActivity`. + +* `NdkNativeActivity` is implemented by the C structure `ANativeActivity`. It + is on the same thread as Nit and manages the synchronization with the + main Android thread. + +* `NativeNativeActivity` is implemented in Java by `android.app.NativeActivity`, + which is a subclass of `Activity` and `Context` (in Java). It represent + main activity of the running application. Use it to get anything related + to the `Context` and as anchor to execute Java UI code. + +## `nit_activity` + +> This module is implemented in 3 languages: + +* The Java code, in `NitActivity.java` acts as the entry point registered + to the Android OS. It relays most of the Android callbacks to C. + In theory, there may be more than one instance of `NitActivity` alive at + a given time. They hold a reference to the corresponding Nit `Activity` + in the attribute `nitActivity`. + +* The C code is defined in the top part of this source file. It acts as a + glue between Java and Nit by relaying calls between both languages. + It keeps a global variables reference to the Java VM and the Nit `App`. + +* The Nit code defines the `Activity` class with the callbacks from Android. + The callback methods should be redefined by user modules. + +The main is invoked when the native library is dynamically linked by +the Java virtual machine. For this reason, the main _must_ execute quickly, +on the main UI thread at least. + +## `notification` + +> ~~~~nitish +~~~~ + +# Create and show a notification + +var notif = new Notification("My Title", "Some content") +notif.ticker = "Ticker text" +notif.show + +# Update the notification + +notif.text = "New content!" +notif.ongoing = true # Make it un-dismissable +notif.show + +# Hide the notification + +notif.cancel + +~~~~ + +For more information, see: +http://developer.android.com/guide/topics/ui/notifiers/notifications.html + +## `sensors` + +> Sensors are to be enabled when `App` is created. +The following example enables all sensors. +The events (`SensorEvent`, `ASensorAccelerometer`, `ASensorMagneticField`...) +are sent to the `input` callback of `App` + +~~~~nitish +redef class App + init + do + sensors_support_enabled = true + accelerometer.enabled = true + accelerometer.eventrate = 10000 + magnetic_field.enabled = true + gyroscope.enabled = true + light.enabled = true + proximity.enabled = true + end +end +~~~~ + +## `audio` + +> `assets` contains the portable version of sounds, since the `res` folder exsists only in android projects. + +For this example, the sounds "test_sound" and "test_music" are located in the "assets/sounds" folder, +they both have ".ogg" extension. "test_sound" is a short sound and "test_music" a music track + +~~~nitish +# Note that you need to specify the path from "assets" folder and the extension +var s = new Sound("sounds/test_sound.ogg") +var m = new Music("sounds/test_music.ogg") +s.play +m.play +~~~ + +Now, the sounds are in "res/raw" + +~~~nitish +s = app.load_sound_from_res("test_sound") +m = app.load_music_from_res("test_sound") +s.play +m.play +~~~ + +See http://developer.android.com/reference/android/media/package-summary.html for more infos + +## Authors + +This project is maintained by **Alexis Laferrière **. diff --git a/lib/app/README.docdown.md b/lib/app/README.docdown.md new file mode 100644 index 0000000000..cfab648f98 --- /dev/null +++ b/lib/app/README.docdown.md @@ -0,0 +1,260 @@ +# `app` - _app.nit_, a framework for portable applications + +[[toc: app]] + +The framework provides services to manage common needs of modern mobile applications: + +* Life-cycle +* User interface +* Persistence +* Async HTTP requests +* Package metadata +* Compilation and packaging + +The features offered by _[[app]].nit_ are common to all platforms, but +may not be available on all devices. + +## Getting Started + +These instructions will get you a copy of the project up and running on your local machine. + +### Dependencies + +This project requires the following packages: + +[[parents: app]] + +### Run `app_base` + +[[uml: app | format: svg, mentities: app::app;app::data_store;app::assets;app::audio;app::app_base;app::http_request;app::ui]] + +Compile `app_base` with the following command: + +[[main-compile: app::app_base]] + +Then run it with: + +[[main-run: app::app_base]] + +## [[sign: app::app]] + +> [[doc: app::app]] + +## Application Life-Cycle + +The _app.nit_ application life-cycle is compatible with all target platforms. +It relies on the following sequence of events, represented here by their callback method name: + +1. `on_create`: The application is being created. + You should build the UI at this time and launch services. + +2. `on_resume`: The app enters the active state, it is in the foreground and interactive. + +3. `on_pause`: The app becomes inactive and it leaves the foreground. + It may still be visible in the background. + +4. `on_stop`: The app is completely hidden. + It may then be destroyed (without warning) or go back to the active state with `on_restart`. + +5. `on_restart`: The app goes back to the inactive state. + You can revert what was done by `on_stop`. + +![_app.nit_ life-cycle](doc/app-nit-lifecycle.png) + +Life-cycle events related to saving and restoring the application state are provided by two special callback methods: + +* `on_save_state`: The app may be destroyed soon, save its state for a future `on_restore_state`. + There is more on how it can be done in the `app::data_store` section. + +* `on_restore_state`: The app is launching, restore its state from a previous `on_save_state`. + +These events are synchronized to the native platforms applications +The [[app::App | text: `App`]] instance is the first to be notified of these events. +Other UI elements, from the [[app::ui | text: `ui`]] submodule, are notified of the same events using a simple depth first visit. +So all UI elements can react separately to live-cycle events. + +## [[sign: app::ui]] + +> [[doc: app::ui]] + +The `app::ui` module defines an abstract API to build a portable graphical application. +The API is composed of interactive `Control`s, visible [[app::View | text: `View`]]s and an active [[app::Window | text: `Window`]]. + +Here is a subset of the most useful controls and views: + +* The classic pushable [[app::Button | text: `Button`]] with text (usually rectangular). + +* [[app::TextInput | text: `TextInput`]] is a field for the user to enter text. + +* [[app::HorizontalLayout | text: `HorizontalLayout`]] and [[app::VerticalLayout | text: `VerticalLayout`]] organize other controls in order. + +Each control is notified of input events by callbacks to `on_event`. +All controls have observers that are also notified of the events. +So there is two ways to customize the behavior on a given event: + +* Create a subclass of the wanted [[app::Control | text: `Control`]], let's say `Button`, and specialize `on_event`. + +* Add an observer to a `Button` instance, and implement `on_event` in the observer. + +### Usage Example + +The example at `examples/ui_example.nit` shows off most features of `app::ui` in a minimal program. +You can also take a look at the calculator (`../../examples/calculator/src/calculator.nit`) which is a concrete usage example. + +### Platform-specific UI + +You can go beyond the portable UI API of _app.nit_ by using the natives services of a platform. + +The suggested approach is to use platform specific modules to customize the application on a precise platform. +See the calculator example for an adaptation of the UI on Android, +the interesting module is in this repository at ../../examples/calculator/src/android_calculator.nit + +## [[sign: app::data_store]] + +> [[doc: app::data_store]] + +_[[app::app]].nit_ offers the submodule [[app::data_store | text: `app::data_store`]] to easily save the application state and user preferences. +The service is accessible by the method `App::data_store`. The [[app::DataStore | text: `DataStore`]] itself defines 2 methods: + +* `DataStore::[]=` saves and associates any serializable instances to a `String` key. + Pass `null` to clear the value associated to a key. + +* `DataStore::[]` returns the object associated to a `String` key. + It returns `null` if nothing is associated to the key. + + Example from `app::http_request_example`: + +[[code: app::http_request_example]] + +### Usage Example + +~~~ +import app::data_store + +redef class App + var user_name: String + + redef fun on_save_state + do + app.data_store["first run"] = false + app.data_store["user name"] = user_name + + super # call `on_save_state` on all attached instances of `AppComponent` + end + + redef fun on_restore_state + do + var first_run = app.data_store["first run"] + if first_run != true then + print "It's the first run of this application" + end + + var user_name = app.data_store["user name"] + if user_name isa String then + self.user_name = user_name + else self.user_name = "Undefined" + + super + end +end +~~~ + +## [[sign: app::http_request]] + +> [[doc: app::http_request]] + +The module [[app::http_request | text: `app::http_request`]] provides services to execute asynchronous HTTP request. +The class [[app::AsyncHttpRequest | text: `AsyncHttpRequest`]] hides the complex parallel logic and +lets the user implement methods acting only on the UI thread. +See the documentation of `AsyncHttpRequest` for more information and +the full example at `examples/http_request_example.nit`. + +Example from `app::ui_example`: + +[[code: app::ui_example]] + +[[features: app | mentities: app::app;app::data_store;app::assets;app::audio;app::app_base;app::http_request;app::ui]] + +## Metadata annotations + +The _app.nit_ framework defines three annotations to customize the application package. + +* `app_name` takes a single argument, the visible name of the application. + This name is used for launchers and window title. + By default, the name of the target module. + +* `app_namespace` specifies the full namespace (or package name) of the application package. + This value usually identify the application uniquely on application stores. + It should not change once the application has benn published. + By default, the namespace is `org.nitlanguage.{module_name}`. + +* `app_version` specifies the version of the application package. + This annotation expects at least one argument, usually we use three version numbers: + the major, minor and revision. + The special function `git_revision` will use the prefix of the hash of the latest git commit. + By default, the version is 0.1. + +* `app_files` tells the compiler where to find platform specific resource files associated to a module. + By default, only the root of the project is searched for the folders `android` and `ios`. + The `android` folder is used as base for the generated Android project, + it can be used to specify the resource files, libs and even Java source files. + The `ios` folder is searched for icons only. + + Each argument of `app_files` is a relative path to a folder containing extra `android` or `ios` folders. + If there is no arguments, the parent folder of the annotated module is used. + In case of name conflicts in the resource files, the files from the project root have the lowest priority, + those associated to modules lower in the importation hierarchy have higher priority. + +### Usage Example + +~~~ +module my_module is + app_name "My App" + app_namespace "org.example.my_app" + app_version(1, 0, git_revision) +end +~~~ + +## Compiling and Packaging an Application + +The Nit compiler detects the target platform from the importations and generates the appropriate application format and package. + +Applications using only the portable services of _app.nit_ require some special care at compilation. +Such an application, let's say `calculator.nit`, does not depend on a specific platform and use the portable UI. +The target platform must be specified to the compiler for it to produce the correct application package. +There is two main ways to achieve this goal: + +* The mixin option (`-m module`) imports an additional module before compiling. + It can be used to load platform specific implementations of the _app.nit_ portable UI. + + ~~~bash + # GNU/Linux version, using GTK + nitc calculator.nit -m linux + + # Android version + nitc calculator.nit -m android + + # iOS version + nitc calculator.nit -m ios + ~~~ + +* A common alternative for larger projects is to use platform specific modules. + Continuing with the calculator example, it is adapted for Android by the module `android_calculator.nit`. + This module imports both `calculator` and `android`, it can then use Android specific code. + + ~~~ + module android_calculator + + import calculator + import android + + # ... + ~~~ + +## [[sign: app::audio]] + +> [[doc: app::audio]] + +## Authors + +This project is maintained by [[ini-maintainer: app]]. diff --git a/lib/app/README.md b/lib/app/README.md index 4baecf4e75..2979c02aae 100644 --- a/lib/app/README.md +++ b/lib/app/README.md @@ -1,4 +1,21 @@ -_app.nit_, a framework for portable applications +# `app` - _app.nit_, a framework for portable applications + +* [Getting Started](#Getting-Started) +* [Dependencies](#Dependencies) +* [Run `app_base`](#Run-`app_base`) +* [`app`](#`app`) +* [Application Life-Cycle](#Application-Life-Cycle) +* [`ui`](#`ui`) +* [Usage Example](#Usage-Example) +* [Platform-specific UI](#Platform-specific-UI) +* [`data_store`](#`data_store`) +* [Usage Example](#Usage-Example) +* [`http_request`](#`http_request`) +* [Metadata annotations](#Metadata-annotations) +* [Usage Example](#Usage-Example) +* [Compiling and Packaging an Application](#Compiling-and-Packaging-an-Application) +* [`audio`](#`audio`) +* [Authors](#Authors) The framework provides services to manage common needs of modern mobile applications: @@ -9,10 +26,45 @@ The framework provides services to manage common needs of modern mobile applicat * Package metadata * Compilation and packaging -The features offered by _app.nit_ are common to all platforms, but +The features offered by _[app](app).nit_ are common to all platforms, but may not be available on all devices. -# Application Life-Cycle +## Getting Started + +These instructions will get you a copy of the project up and running on your local machine. + +### Dependencies + +This project requires the following packages: + +* `android` - Android platform support and APIs +* `core` - Nit common library of core classes and methods +* `json` - read and write JSON formatted text +* `pthreads` - POSIX Threads support +* `serialization` - Abstract serialization services + +### Run `app_base` + +![Diagram for `app`](uml-app.svg) + +Compile `app_base` with the following command: + +~~~bash +nitc ./app_base.nit +~~~ + +Then run it with: + +~~~bash +./app_base +~~~ + +## `app` + +> The features offered by this modules are common to all platforms, but +> may not be available on all devices. + +## Application Life-Cycle The _app.nit_ application life-cycle is compatible with all target platforms. It relies on the following sequence of events, represented here by their callback method name: @@ -41,37 +93,53 @@ Life-cycle events related to saving and restoring the application state are prov * `on_restore_state`: The app is launching, restore its state from a previous `on_save_state`. These events are synchronized to the native platforms applications -The `App` instance is the first to be notified of these events. -Other UI elements, from the `ui` submodule, are notified of the same events using a simple depth first visit. +The [`App`](app::App) instance is the first to be notified of these events. +Other UI elements, from the [`ui`](app::ui) submodule, are notified of the same events using a simple depth first visit. So all UI elements can react separately to live-cycle events. -# User Interface +## `ui` + +> ~~~ +import app::ui + +class MyWindow + super Window + + var layout = new ListLayout(parent=self) + var lbl = new Label(parent=layout, text="Hello world", align=0.5) + var but = new Button(parent=layout, text="Press here") + + redef fun on_event(event) do lbl.text = "Pressed!" +end + +redef fun root_window do return new MyWindow +~~~ The `app::ui` module defines an abstract API to build a portable graphical application. -The API is composed of interactive `Control`s, visible `View`s and an active `Window`. +The API is composed of interactive `Control`s, visible [`View`](app::View)s and an active [`Window`](app::Window). Here is a subset of the most useful controls and views: -* The classic pushable `Button` with text (usually rectangular). +* The classic pushable [`Button`](app::Button) with text (usually rectangular). -* `TextInput` is a field for the user to enter text. +* [`TextInput`](app::TextInput) is a field for the user to enter text. -* `HorizontalLayout` and `VerticalLayout` organize other controls in order. +* [`HorizontalLayout`](app::HorizontalLayout) and [`VerticalLayout`](app::VerticalLayout) organize other controls in order. Each control is notified of input events by callbacks to `on_event`. All controls have observers that are also notified of the events. So there is two ways to customize the behavior on a given event: -* Create a subclass of the wanted `Control`, let's say `Button`, and specialize `on_event`. +* Create a subclass of the wanted [`Control`](app::Control), let's say `Button`, and specialize `on_event`. * Add an observer to a `Button` instance, and implement `on_event` in the observer. -## Usage Example +### Usage Example The example at `examples/ui_example.nit` shows off most features of `app::ui` in a minimal program. You can also take a look at the calculator (`../../examples/calculator/src/calculator.nit`) which is a concrete usage example. -## Platform-specific UI +### Platform-specific UI You can go beyond the portable UI API of _app.nit_ by using the natives services of a platform. @@ -79,18 +147,97 @@ The suggested approach is to use platform specific modules to customize the appl See the calculator example for an adaptation of the UI on Android, the interesting module is in this repository at ../../examples/calculator/src/android_calculator.nit -# Persistent State with data\_store +## `data_store` + +> The main services is `App::data_store`, a `DataStore` holding any +> serializable Nit object. _app.nit_ offers the submodule `app::data_store` to easily save the application state and user preferences. The service is accessible by the method `App::data_store`. The `DataStore` itself defines 2 methods: * `DataStore::[]=` saves and associates any serializable instances to a `String` key. -Pass `null` to clear the value associated to a key. + Pass `null` to clear the value associated to a key. * `DataStore::[]` returns the object associated to a `String` key. -It returns `null` if nothing is associated to the key. + It returns `null` if nothing is associated to the key. + + Example from `app::http_request_example`: + +~~~ +# Example for the `app::http_request` main service `AsyncHttpRequest` +module http_request_example is + example + app_name "app.nit HTTP" + app_namespace "org.nitlanguage.http_example" + android_api_target 15 +end + +import app::ui +import app::http_request +import android::aware # for android_api_target + +# Simple asynchronous HTTP request to http://example.com/ displaying feedback to the window +class MyHttpRequest + super AsyncHttpRequest + + # Back reference to the window to show feedback to the user + var win: HttpRequestClientWindow + + # --- + # Config the request + + redef fun uri do return "http://example.com/" + redef fun deserialize_json do return false + + # --- + # Customize callbacks + + redef fun before + do + win.label_response.text = "Sending request..." + + # Disable button to prevent double requests + win.button_request.enabled = false + end + + redef fun on_load(data, status) + do win.label_response.text = "Received response code {status} with {data.as(Text).byte_length} bytes" -## Usage Example + redef fun on_fail(error) + do win.label_response.text = "Connection error: {error}" + + redef fun after do win.button_request.enabled = true +end + +# Simple window with a label and a button +class HttpRequestClientWindow + super Window + + # Root layout + var layout = new ListLayout(parent=self) + + # Button to send request + var button_request = new Button(parent=layout, text="Press to send HTTP request") + + # Label displaying feedback to user + var label_response = new Label(parent=layout, text="No response yet.") + + init do button_request.observers.add self + + redef fun on_event(event) + do + if event isa ButtonPressEvent and event.sender == button_request then + # Prepare and send request + var request = new MyHttpRequest(self) + request.start + end + end +end + +redef fun root_window do return new HttpRequestClientWindow +~~~ + +### Usage Example ~~~ import app::data_store @@ -123,15 +270,123 @@ redef class App end ~~~ -# Async HTTP request +## `http_request` + +> ~~~nitish +~~~ + +import app::http_request + +class MyHttpRequest +super AsyncHttpRequest + + redef fun uri do return "http://example.com/" + + redef fun on_load(data, status) do print "Received: {data or else "null"}" + + redef fun on_fail(error) do print "Connection error: {error}" + +end + +var req = new MyHttpRequest +req.start -The module `app::http_request` provides services to execute asynchronous HTTP request. -The class `AsyncHttpRequest` hides the complex parallel logic and +~~~ + +The module [`app::http_request`](app::http_request) provides services to execute asynchronous HTTP request. +The class [`AsyncHttpRequest`](app::AsyncHttpRequest) hides the complex parallel logic and lets the user implement methods acting only on the UI thread. See the documentation of `AsyncHttpRequest` for more information and the full example at `examples/http_request_example.nit`. -# Metadata annotations +Example from `app::ui_example`: + +~~~ +# User interface example using `app::ui` +module ui_example is + example + app_name "app.nit UI" + app_namespace "org.nitlanguage.ui_example" + android_api_min 21 + android_api_target 21 + android_manifest_activity "android:theme=\"@android:style/Theme.Material\"" +end + +import app::ui +import app::data_store +import android::aware # for android_api_target + +# Window showing off some the available controls +class UiExampleWindow + super Window + + # Root layout + var layout = new ListLayout(parent=self) + + # Some label + var some_label = new Label(parent=layout, text="Sample Window using a ListLayout.") + + # A checkbox + var checkbox = new CheckBox(parent=layout, text="A CheckBox") + + # Horizontal organization + var h_layout = new HorizontalLayout(parent=layout) + + # Description for the `user_input` + var user_input_label = new Label(parent=h_layout, text="Input some text:", align=0.5) + + # Field for the user to enter data + var user_input = new TextInput(parent=h_layout, text="Default text") + + # Button to open a new window with a ListLayout + var button_window = new Button(parent=layout, text="Open a new window") + + # URL to open + var example_url = "http://nitlanguage.org/" + + # Button to open the browser + var button_browser = new Button(parent=layout, text="Open {example_url}") + + redef fun on_event(event) + do + if event isa ButtonPressEvent then + if event.sender == button_browser then + example_url.open_in_browser + else if event.sender == button_window then + app.push_window new SecondWindow + end + else if event isa ToggleEvent then + if event.sender == checkbox then checkbox.text = if checkbox.is_checked then "Checked" else "Unchecked" + end + end +end + +# Another window with a small `VerticalLayout` +class SecondWindow + super Window + + # Root layout + var layout = new VerticalLayout(parent=self) + + # Some label + var a_label = new Label(parent=layout, text="This window uses a VerticalLayout.") + + # Another label + var another_label = new Label(parent=layout, text="Close it by tapping the back button.") +end + +redef fun root_window do return new UiExampleWindow +~~~ + +* `app` - app.nit is a framework to create cross-platform applications +* `app_base` - Base of the _app.nit_ framework, defines `App` +* `assets` - Portable services to load resources from the assets folder +* `audio` - Services to load and play `Sound` and `Music` from the assets folder +* `data_store` - Key/value storage services +* `http_request` - HTTP request services: `AsyncHttpRequest` and `Text::http_get` +* `ui` - Portable UI controls for mobiles apps + +## Metadata annotations The _app.nit_ framework defines three annotations to customize the application package. @@ -161,7 +416,7 @@ The _app.nit_ framework defines three annotations to customize the application p In case of name conflicts in the resource files, the files from the project root have the lowest priority, those associated to modules lower in the importation hierarchy have higher priority. -## Usage Example +### Usage Example ~~~ module my_module is @@ -171,7 +426,7 @@ module my_module is end ~~~ -# Compiling and Packaging an Application +## Compiling and Packaging an Application The Nit compiler detects the target platform from the importations and generates the appropriate application format and package. @@ -183,7 +438,7 @@ There is two main ways to achieve this goal: * The mixin option (`-m module`) imports an additional module before compiling. It can be used to load platform specific implementations of the _app.nit_ portable UI. - ~~~ + ~~~bash # GNU/Linux version, using GTK nitc calculator.nit -m linux @@ -206,3 +461,13 @@ There is two main ways to achieve this goal: # ... ~~~ + +## `audio` + +> Get a handle to a sound using `new Sound` or `new Music` at any time. +> Call `load` at or after `App::on_create` or leave it to be loaded +> on demand by the first call to `play`. + +## Authors + +This project is maintained by **Alexis Laferrière **. diff --git a/lib/core/README.docdown.md b/lib/core/README.docdown.md new file mode 100644 index 0000000000..cb2aee3469 --- /dev/null +++ b/lib/core/README.docdown.md @@ -0,0 +1,95 @@ +# `core` - Nit common library of core classes and methods + +[[toc: core]] + +[[core | text: Core]] classes and methods used by default by Nit programs and libraries. + +[[features: core | mentities: core::codecs;core::text;core::bitset;core::gc;core::codec_base;core::core;core::error;core::environ;core::protocol;core::iso8859_1;core::utf8;core::collection;core::numeric;core::range;core::time;core::exec;core::union_find;core::math;core::queue;core::fixed_ints_text;core::hash_collection;core::sorter;core::circular_array;core::native;core::list;core::re;core::abstract_collection;core::string_search;core::kernel;core::stream;core::fixed_ints;core::array;core::ropes;core::file;core::bytes;core::abstract_text;core::flat]] + +## Core Basic Types and Operations + +[[uml: core | format: svg, mentities: core::codecs;core::text;core::bitset;core::gc;core::codec_base;core::core;core::error;core::environ;core::protocol;core::iso8859_1;core::utf8;core::collection;core::numeric;core::range;core::time;core::exec;core::union_find;core::math;core::queue;core::fixed_ints_text;core::hash_collection;core::sorter;core::circular_array;core::native;core::list;core::re;core::abstract_collection;core::string_search;core::kernel;core::stream;core::fixed_ints;core::array;core::ropes;core::file;core::bytes;core::abstract_text;core::flat]] + +## [[sign: core::kernel]] + +> [[doc: core::kernel]] + +### [[sign: core::Object]] + +> [[doc: core::Object]] + +#### Equality + +[[doc: core::Object::==]] +[[doc: core::Object::!=]] +[[doc: core::Object::hash]] +[[doc: core::Object::is_same_instance]] +[[doc: core::Object::object_id]] + +#### Debuging + +[[doc: core::Object::output]] +[[doc: core::Object::output_class_name]] +[[doc: core::Object::is_same_type]] + +### [[sign: core::Sys]] + +> [[doc: core::Sys]] + +#### Program Execution + +[[doc: core::Sys::main]] +[[doc: core::Sys::run]] + +### Other + +[[defs: kernel]] + +## Core Collections + +[[doc: core::collection]] + +### [[sign: core::abstract_collection]] + +> [[doc: core::abstract_collection]] + +### [[sign: core::queue]] + +> [[doc: core::queue]] + +### [[sign: core::sorter]] + +> [[doc: core::sorter]] + +## String and Text manipulation + +[[doc: core::text]] + +### [[sign: core::ropes]] + +> [[doc: core::ropes]] + +## [[sign: core::codec_base]] + +> [[doc: core::codec_base]] + +## [[sign: core::re]] + +> [[doc: core::re]] + +## [[sign: core::fixed_ints]] + +> [[doc: core::fixed_ints]] + +## Running the tests + +Run the nitunit automated tests with the following command: + +[[testing: core]] + +## Authors + +This project is maintained by [[ini-maintainer: core]]. + +Thanks to the contribution of: +[[ini-contributors: core]] diff --git a/lib/core/README.md b/lib/core/README.md index 2501758b02..c75e403e8b 100644 --- a/lib/core/README.md +++ b/lib/core/README.md @@ -1,46 +1,420 @@ -# Nit common library of core classes and methods +# `core` - Nit common library of core classes and methods -Core classes and methods used by default by Nit programs and libraries. +* [Core Basic Types and Operations](#Core-Basic-Types-and-Operations) +* [`kernel`](#`kernel`) +* [`Object`](#`Object`) +* [Equality](#Equality) +* [Debuging](#Debuging) +* [`Sys`](#`Sys`) +* [Program Execution](#Program-Execution) +* [Other](#Other) +* [Core Collections](#Core-Collections) +* [`abstract_collection`](#`abstract_collection`) +* [`queue`](#`queue`) +* [`sorter`](#`sorter`) +* [String and Text manipulation](#String-and-Text-manipulation) +* [`ropes`](#`ropes`) +* [`codec_base`](#`codec_base`) +* [`re`](#`re`) +* [`fixed_ints`](#`fixed_ints`) +* [Running the tests](#Running-the-tests) +* [Authors](#Authors) + +[Core](core) classes and methods used by default by Nit programs and libraries. + +* `abstract_collection` - Abstract collection classes and services. +* `abstract_text` - Abstract class for manipulation of sequences of characters +* `array` - This module introduces the standard array structure. +* `bitset` - Services to handle BitSet +* `bytes` - Services for byte streams and arrays +* `circular_array` - Efficient data structure to access both end of the sequence. +* `codec_base` - Base for codecs to use with streams +* `codecs` - Group module for all codec-related manipulations +* `collection` - This module define several collection classes. +* `core` - Standard classes and methods used by default by Nit programs and libraries. +* `environ` - Access to the environment variables of the process +* `error` - Standard error-management infrastructure. +* `exec` - Invocation and management of operating system sub-processes. +* `file` - File manipulations (create, read, write, etc.) +* `fixed_ints` - Basic integers of fixed-precision +* `fixed_ints_text` - Text services to complement `fixed_ints` +* `flat` - All the array-based text representations +* `gc` - Access to the Nit internal garbage collection mechanism +* `hash_collection` - Introduce `HashMap` and `HashSet`. +* `iso8859_1` - Codec for ISO8859-1 I/O +* `kernel` - Most basic classes and methods. +* `list` - This module handle double linked lists +* `math` - Mathematical operations +* `native` - Native structures for text and bytes +* `numeric` - Advanced services for `Numeric` types +* `protocol` +* `queue` - Queuing data structures and wrappers +* `range` - Module for range of discrete objects. +* `re` - Regular expression support for all services based on `Pattern` +* `ropes` - Tree-based representation of a String. +* `sorter` - This module contains classes used to compare things and sorts arrays. +* `stream` - Input and output streams of characters +* `string_search` - Basic string search, match and replace. +* `text` - All the classes and methods related to the manipulation of text entities +* `time` - Management of time and dates +* `union_find` - union–find algorithm using an efficient disjoint-set data structure +* `utf8` - Codec for UTF-8 I/O ## Core Basic Types and Operations -[[doc: kernel]] +![Diagram for `core`](uml-core.svg) + +## `kernel` -### Object +> This module is the root of the module hierarchy. +> It provides a very minimal set of classes and services used as a +> foundation to define other classes and methods. -[[doc: Object]] +### `Object` + +> Each other class implicitly specializes Object, +> therefore the services of Object are inherited by every other class and are usable +> on each value, including primitive types like integers (`Int`), strings (`String`) and arrays (`Array`). + +Note that `nullable Object`, not `Object`, is the root of the type hierarchy +since the special value `null` is not considered as an instance of Object. #### Equality -[[doc: core::Object::==]] -[[doc: core::Object::!=]] -[[doc: core::Object::hash]] -[[doc: core::Object::is_same_instance]] -[[doc: core::Object::object_id]] +~~~ +assert 1 + 1 == 2 +assert not 1 == "1" +assert 1.to_s == "1" +~~~ + +The exact meaning of *same value* is left to the subclasses. +Implicitly, the default implementation, is `is_same_instance`. + +The laws of `==` are the following: + +* reflexivity `a.is_same_instance(b) implies a == b` +* symmetry: `(a == b) == (b == a)` +* transitivity: `(a == b) and (b == c) implies (a == c)` + +`==` might not be constant on some objects overtime because of their evolution. + +~~~ +var a = [1] +var b = [1] +var c = [1,2] +assert a == b and not a == c +a.add 2 +assert not a == b and a == c +~~~ + +Lastly, `==` is highly linked with `hash` and a specific redefinition of `==` should +usually be associated with a specific redefinition of `hash`. + +ENSURE `result implies self.hash == other.hash` +`!=` is equivalent with `not ==`. +The hash code is used in many data-structures and algorithms to identify objects that might be equal. +Therefore, the precise semantic of `hash` is highly linked with the semantic of `==` +and the only law of `hash` is that `a == b implies a.hash == b.hash`. + +~~~ +assert (1+1).hash == 2.hash +assert 1.to_s.hash == "1".hash +~~~ + +`hash` (like `==`) might not be constant on some objects over time because of their evolution. + +~~~ +var a = [1] +var b = [1] +var c = [1,2] +assert a.hash == b.hash +a.add 2 +assert a.hash == c.hash +# There is a very high probability that `b.hash != c.hash` +~~~ + +A specific redefinition of `==` should usually be associated with a specific redefinition of `hash`. +Note that, unfortunately, a correct definition of `hash` that is lawful with `==` is sometime tricky +and a cause of bugs. + +Without redefinition, `hash` is based on the `object_id` of the instance. + +~~~ +var a = new Buffer +var b = a +var c = new Buffer +assert a.is_same_instance(b) +assert not a.is_same_instance(c) +assert a == c # because both buffers are empty +~~~ + +Obviously, the identity of an object is preserved even if the object is mutated. + +~~~ +var x = [1] +var y = x +x.add 2 +assert x.is_same_instance(y) +~~~ + +Unless specific code, you should use `==` instead of `is_same_instance` because +most of the time is it the semantic (and user-defined) comparison that make sense. + +Moreover, relying on `is_same_instance` on objects you do not control +might have unexpected effects when libraries reuse objects or intern them. +Unless specific code, you should not use this method but +use `hash` instead. + +As its name hints it, the internal hash code, is used internally +to provide a hash value. +It is also used by the `inspect` method to loosely identify objects +and helps debugging. + +~~~ +var a = "Hello" +var b = a +assert a.object_id == b.object_id +~~~ + +The specific details of the internal hash code it let to the specific +engine. The rules are the following: + +* The `object_id` MUST be invariant for the whole life of the object. +* Two living instances of the same classes SHOULD NOT share the same `object_id`. +* Two instances of different classes MIGHT share the same `object_id`. +* The `object_id` of a garbage-collected instance MIGHT be reused by new instances. +* The `object_id` of an object MIGHT be non constant across different executions. + +For instance, the `nitc` compiler uses the address of the object in memory +as its `object_id`. + +TODO rename in something like `internal_hash_code` #### Debuging -[[doc: core::Object::output]] -[[doc: core::Object::output_class_name]] -[[doc: core::Object::is_same_type]] +This method MUST not be used by programs, it is here for debugging +only and can be removed without any notice. + +TODO: rename to avoid blocking a good identifier like `output`. +This method MUST not be used by programs, it is here for debugging +only and can be removed without any notice. + +TODO: rename to avoid blocking a good identifier like `output`. + +~~~ +assert 1.is_same_type(2) +assert "Hello".is_same_type("World") +assert not "Hello".is_same_type(2) +~~~ + +The method returns false if the dynamic type of `other` is a subtype of the dynamic type of `self` +(or the other way around). -### Sys +Unless specific code, you should not use this method because it is inconsistent +with the fact that a subclass can be used in lieu of a superclass. -[[doc: Sys]] +### `Sys` + +> `Sys` is a singleton class, its only instance is accessible from everywhere with `sys`. + +Because of this, methods that should be accessible from everywhere, like `print` or `exit`, +are defined in `Sys`. +Moreover, unless there is an ambiguity with `self`, the receiver of a call to these methods is implicitly `sys`. +Basically it means that the two following instructions are equivalent. + +~~~nit +print "Hello World" +sys.print "Hello World" +~~~ + +## Methods Implicitly Defined in Sys + +`Sys` is the class where are defined top-level methods, +i.e. those defined outside of any class like in a procedural language. +Basically it means that + +~~~nitish +redef class Sys + fun foo do print "hello" +end +~~~ + +is equivalent with + +~~~nitish +fun foo print "hello" +~~~ + +As a corollary, in a top-level method, `self` (the current receiver) is always `sys`. #### Program Execution -[[doc: core::Sys::main]] -[[doc: core::Sys::run]] +In a module, the instructions defined outside any classes or methods +(usually called the *main* of the module) is +an implicit definition of this `main` method. +Basically it means that the following program + +~~~nit +print "Hello World" +~~~ + +is equivalent with + +~~~nit +redef class Sys + redef fun main do + print "Hello World" + end +end +~~~ + +When a program starts, the following implicit sequence of instructions is executed + +~~~nitish +sys = new Sys +sys.run +~~~ + +Whereas the job of the `run` method is just to execute `main`. + +The only reason of the existence of `run` is to allow modules to refine it +and inject specific work before or after the main part. ### Other -[[defs: kernel]] +* `kernel$Bool` - Native Booleans. +* `kernel$Byte` - Native bytes. +* `kernel$Char` - Native characters. +* `kernel$Cloneable` - Something that can be cloned +* `kernel$Comparable` - The ancestor of class where objects are in a total order. +* `kernel$Discrete` - Discrete total orders. +* `kernel$Float` - Native floating point numbers. +* `kernel$Int` - Native integer numbers. +* `kernel$Numeric` - A numeric value supporting mathematical operations +* `kernel$Object` - The root of the class hierarchy. +* `kernel$Pointer` - Pointer classes are used to manipulate extern C structures. +* `kernel$Sys` - The main class of the program. +* `kernel$Task` - Task with a `main` method to be implemented by subclasses ## Core Collections -[[doc: core::collection]] +### `abstract_collection` + +> TODO specify the behavior on iterators when collections are modified. + +### `queue` + +> Topics: + +* `Queue` +* `Sequence::as_lifo` +* `Sequence::as_fifo` +* `SimpleCollection::as_random` +* `MinHeap` + +### `sorter` + +> In order to provide your own sort class you should define a subclass of `Comparator` with +> a custom `Comparator::compare` function and a specific `COMPARED` virtual type. ## String and Text manipulation -[[doc: core::text]] +### `ropes` + +> Ropes are a data structure introduced in a 1995 paper from +> Hans J. Boehm, Russ Atkinson and Michael Plass. + +See: + +> Ropes: an Alternative to Strings, +> *Software - Practice and Experience*, +> Vol. 25(12), 1315-1330 (December 1995). + +The implementation developed here provides an automatic change +of data structure depending on the length of the leaves. + +The length from which a `Rope` is built from a `flat` string +if defined at top-level (see `maxlen`) and can be redefined by clients +depending on their needs (e.g. if you need to bench the speed of +the creation of concat nodes, lower the size of maxlen). + +A Rope as defined in the original paper is a Tree made of concatenation +nodes and containing `Flat` (that is Array-based) strings as Leaves. + +Example : + +~~~raw + Concat + / \ + Concat Concat + / \ / \ + "My" " Name" " is" " Simon." +~~~ + +Note that the above example is not representative of the actual implementation +of `Ropes`, since short leaves are merged to keep the rope at an acceptable +height (hence, this rope here might actually be a `FlatString` !). + +## `codec_base` + +> A Codec (Coder/Decoder) is a tranformer from a byte-format to another + +As Nit Strings are UTF-8, a codec works as : + +- Coder: From a UTF-8 string to a specified format (writing) +- Decoder: From a specified format to a UTF-8 string (reading) + +## `re` + +> Implemented using libc regular expressions. + +The main entities are `Text::to_re` and `Regex`. + +## `fixed_ints` + +> All classes defined here have C-equivalents and the semantics of their +> operations are the same as C's + +* Int8 => int8_t +* Int16 => int16_t +* UInt16 => uint16_t +* Int32 => int32_t +* UInt32 => uint32_t + +NOTE: No UInt8 is provided as Byte is the same +SEE: kernel::Byte + +HOW TO USE: +All classes can be instanciated via a literal rule. +Namely, a suffix to append after the literal integer. + +* Int8 => i8 +* Byte => u8 +* Int16 => i16 +* UInt16 => u16 +* Int32 => i32 +* UInt32 => u32 + +## Running the tests + +Run the nitunit automated tests with the following command: + +~~~bash +nitunit . +~~~ + +## Authors + +This project is maintained by **Jean Privat **. + +Thanks to the contribution of: + +* **Jean-Christophe Beaupré ** +* **Romain Chanoir ** +* **Christophe Gigax ** +* **Frédéric Vachon ** +* **Jean-Sebastien Gelinas ** +* **Alexandre Blondin Massé ** +* **Johan Kayser ** +* **Johann Dubois ** +* **Julien Pagès ** diff --git a/lib/gamnit/README.docdown.md b/lib/gamnit/README.docdown.md new file mode 100644 index 0000000000..6501d9e015 --- /dev/null +++ b/lib/gamnit/README.docdown.md @@ -0,0 +1,149 @@ +# `gamnit` - Portable game and multimedia framework for Nit + +[[toc: gamnit]] + +_gamnit_ is a modular framework to create portable 2D or 3D apps in Nit. +It is based on the portability framework _app.nit_ and the OpenGL ES 2.0 standard. + +## Getting Started + +These instructions will get you a copy of the project up and running on your local machine. + +### Dependencies + +This project requires the following packages: + +[[parents: gamnit]] + +### Run `texture_atlas_parser` + +[[uml: gamnit | format: svg, mentities: gamnit::landscape;gamnit::portrait;gamnit::common;gamnit::camera_control;gamnit::network;gamnit::vr;gamnit::android19;gamnit::gamnit_ios;gamnit::display;gamnit::flat;gamnit::keys;gamnit::depth_core;gamnit::limit_fps;gamnit::camera_control_linux;gamnit::input_ios;gamnit::display_ios;gamnit::gamnit;gamnit::display_android;gamnit::font;gamnit::cardboard;gamnit::stereoscopic_view;gamnit::texture_atlas_parser;gamnit::client;gamnit::cameras_cache;gamnit::depth;gamnit::mtl;gamnit::gamnit_linux;gamnit::camera_control_android;gamnit::display_linux;gamnit::server;gamnit::particles;gamnit::tileset;gamnit::virtual_gamepad_spritesheet;gamnit::obj;gamnit::more_lights;gamnit::gamnit_android;gamnit::model_parser_base;gamnit::model_dimensions;gamnit::textures;gamnit::egl;gamnit::selection;gamnit::more_meshes;gamnit::dynamic_resolution;gamnit::virtual_gamepad;gamnit::programs;gamnit::cameras;gamnit::bmfont;gamnit::shadow;gamnit::more_models;gamnit::more_materials;gamnit::flat_core]] + +Compile `texture_atlas_parser` with the following command: + +[[main-compile: gamnit::texture_atlas_parser]] + +Then run it with: + +[[main-run: gamnit::texture_atlas_parser]] + +## System configuration + +To compile the _gamnit_ apps packaged with the Nit repository on GNU/Linux you need to install the dev version of a few libraries and some tools. +On Debian 8.2, this command should install everything needed: + +~~~bash +apt-get install libgles2-mesa-dev libsdl2-dev libsdl2-image-dev libsdl2-mixer-dev inkscape +~~~ + +On Windows 64 bits, using msys2, you can install the required packages with: + +~~~bash +pacman -S mingw-w64-x86_64-angleproject-git mingw-w64-x86_64-SDL2 mingw-w64-x86_64-SDL2_image mingw-w64-x86_64-SDL2_mixer +~~~ + +While macOS isn't supported, it can create iOS apps. +You need to install and setup Xcode, and you may install the GLSL shader validation tool via `brew`: + +~~~bash +brew install glslang +~~~ + +## Services by submodules + +_[[gamnit::gamnit]]_ is modular, different services of the framework are available through different submodules: + +* The main entrypoint [[gamnit | text: `gamnit`]] provides low-level abstractions over some services of OpenGL ES 2.0, like [[gamnit::Texture | text: textures]], [[gamnit::Shader | text: shaders]] and programs. + It defines the basic methods to implement in order to obtain a working game: + `App::frame_core` to update the screen and `App::accept_event` to receive user inputs. + +* [[gamnit>flat> | text: `flat`]] provides an easy to use API for 2D games based on sprites. + Clients of this API redefine `App::update` to update the game logic and fill lists of sprites with objects to draw. + + `App::sprites` lists the sprites of the game world, they are drawn form the point of view of the `world_camera`. + This [[gamnit::Camera | text: camera]] can be moved around in the world by updating the x and y of its `position`, + and the zoom can easily be set using `reset_depth(desired_world_units_on_y)` or the `z` of its `position` + + `App::ui_sprites` lists the UI sprites drawn from the point of view of `ui_camera`. + By default, this camera is pixel-perfect with the origin in the top left corner of the window. + However to support screens with different DPI, it is recommended to standardize + the display size using `reset_depth(height_of_reference_display)`. + +* [[gamnit>depth> | text: `depth`]] defines an API for 3D games based on instances of `Actor`. + + This framework is build upon `flat`, see the doc of this submodule first (just above). + As such, clients should still implement `update` with the game logic, and fill `ui_sprites` for UI elements. + + At each frame, elements in the list `actors` are drawn to screen. + Each `Actor` is composed of a `Model` and other information specific to this instance: + position in the world, rotation and scaling. + +* `limit_fps` refines existing services of _gamnit_ to limit the framerate to a customizable value. + +* `keys` provides `app.pressed_keys` keeping track of the currently pressed keys. + +* [[gamnit>model_parsers> | text: `model_parsers`]] provides services to read and parse models from the [[gamnit>virtual_gamepad>assets> | text: asset]] folder. + +* [[gamnit>network> | text: `network`]] provides a simple communication framework for multiplayer client/server games. + + [[features: gamnit | mentities: gamnit::landscape;gamnit::portrait;gamnit::common;gamnit::camera_control;gamnit::network;gamnit::vr;gamnit::android19;gamnit::gamnit_ios;gamnit::display;gamnit::flat;gamnit::keys;gamnit::depth_core;gamnit::limit_fps;gamnit::camera_control_linux;gamnit::input_ios;gamnit::display_ios;gamnit::gamnit;gamnit::display_android;gamnit::font;gamnit::cardboard;gamnit::stereoscopic_view;gamnit::texture_atlas_parser;gamnit::client;gamnit::cameras_cache;gamnit::depth;gamnit::mtl;gamnit::gamnit_linux;gamnit::camera_control_android;gamnit::display_linux;gamnit::server;gamnit::particles;gamnit::tileset;gamnit::virtual_gamepad_spritesheet;gamnit::obj;gamnit::more_lights;gamnit::gamnit_android;gamnit::model_parser_base;gamnit::model_dimensions;gamnit::textures;gamnit::egl;gamnit::selection;gamnit::more_meshes;gamnit::dynamic_resolution;gamnit::virtual_gamepad;gamnit::programs;gamnit::cameras;gamnit::bmfont;gamnit::shadow;gamnit::more_models;gamnit::more_materials;gamnit::flat_core]] + +## [[sign: gamnit::network]] + +> [[doc: gamnit::network]] + +## [[sign: gamnit::android19]] + +> [[doc: gamnit::android19]] + +## [[sign: gamnit::flat]] + +> [[doc: gamnit::flat]] + +## [[sign: gamnit::keys]] + +> [[doc: gamnit::keys]] + +## [[sign: gamnit::display_android]] + +> [[doc: gamnit::display_android]] + +## [[sign: gamnit::cardboard]] + +> [[doc: gamnit::cardboard]] + +## [[sign: gamnit::client]] + +> [[doc: gamnit::client]] + +## [[sign: gamnit::server]] + +> [[doc: gamnit::server]] + +## [[sign: gamnit::particles]] + +> [[doc: gamnit::particles]] + +## [[sign: gamnit::selection]] + +> [[doc: gamnit::selection]] + +## [[sign: gamnit::dynamic_resolution]] + +> [[doc: gamnit::dynamic_resolution]] + +## [[sign: gamnit::virtual_gamepad]] + +> [[doc: gamnit::virtual_gamepad]] + +## [[sign: gamnit::bmfont]] + +> [[doc: gamnit::bmfont]] + +## [[sign: gamnit::shadow]] + +> [[doc: gamnit::shadow]] + +## Authors + +This project is maintained by [[ini-maintainer: gamnit]]. diff --git a/lib/gamnit/README.md b/lib/gamnit/README.md index 68370d96bf..2827ba6893 100644 --- a/lib/gamnit/README.md +++ b/lib/gamnit/README.md @@ -1,43 +1,109 @@ -Portable game and multimedia framework for Nit +# `gamnit` - Portable game and multimedia framework for Nit + +* [Getting Started](#Getting-Started) +* [Dependencies](#Dependencies) +* [Run `texture_atlas_parser`](#Run-`texture_atlas_parser`) +* [System configuration](#System-configuration) +* [Services by submodules](#Services-by-submodules) +* [`network`](#`network`) +* [`android19`](#`android19`) +* [`flat`](#`flat`) +* [`keys`](#`keys`) +* [`display_android`](#`display_android`) +* [`cardboard`](#`cardboard`) +* [`client`](#`client`) +* [`server`](#`server`) +* [`particles`](#`particles`) +* [`selection`](#`selection`) +* [`dynamic_resolution`](#`dynamic_resolution`) +* [`virtual_gamepad`](#`virtual_gamepad`) +* [`bmfont`](#`bmfont`) +* [`shadow`](#`shadow`) +* [Authors](#Authors) _gamnit_ is a modular framework to create portable 2D or 3D apps in Nit. It is based on the portability framework _app.nit_ and the OpenGL ES 2.0 standard. -# System configuration +## Getting Started + +These instructions will get you a copy of the project up and running on your local machine. + +### Dependencies + +This project requires the following packages: + +* `android` - Android platform support and APIs +* `app` - _app.nit_, a framework for portable applications +* `core` - Nit common library of core classes and methods +* `dom` - Easy XML DOM parser +* `egl` - Interface between rendering APIs (OpenGL, OpenGL ES, etc.) and the native windowing system. +* `gen_nit` - Support to generate and otherwise manipulate Nit code +* `geometry` - Basic geometry data structures and services +* `glesv2` - OpenGL graphics rendering library for embedded systems, version 2.0 +* `ios` - iOS support for _app.nit_ +* `linux` - Implementation of app.nit for the Linux platform +* `matrix` - Services for matrices of `Float` values +* `mnit` +* `more_collections` - Highly specific, but useful, collections-related classes. +* `msgpack` - MessagePack, an efficient binary serialization format +* `opts` - Management of options on the command line +* `parser_base` - Simple base for hand-made parsers of all kinds +* `performance_analysis` - Services to gather information on the performance of events by categories +* `realtime` - Services to keep time of the wall clock time +* `sdl2` - Low-level wrapper of the SDL 2.0 library (as `sdl2`) and SDL_image 2.0 (as `sdl2::image`) +* `socket` - Socket services + +### Run `texture_atlas_parser` + +![Diagram for `gamnit`](uml-gamnit.svg) + +Compile `texture_atlas_parser` with the following command: + +~~~bash +nitc ./texture_atlas_parser.nit +~~~ + +Then run it with: + +~~~bash +./texture_atlas_parser +~~~ + +## System configuration To compile the _gamnit_ apps packaged with the Nit repository on GNU/Linux you need to install the dev version of a few libraries and some tools. On Debian 8.2, this command should install everything needed: -~~~ +~~~bash apt-get install libgles2-mesa-dev libsdl2-dev libsdl2-image-dev libsdl2-mixer-dev inkscape ~~~ On Windows 64 bits, using msys2, you can install the required packages with: -~~~ +~~~bash pacman -S mingw-w64-x86_64-angleproject-git mingw-w64-x86_64-SDL2 mingw-w64-x86_64-SDL2_image mingw-w64-x86_64-SDL2_mixer ~~~ While macOS isn't supported, it can create iOS apps. You need to install and setup Xcode, and you may install the GLSL shader validation tool via `brew`: -~~~ +~~~bash brew install glslang ~~~ -# Services by submodules +## Services by submodules -_gamnit_ is modular, different services of the framework are available through different submodules: +_[gamnit](gamnit::gamnit)_ is modular, different services of the framework are available through different submodules: -* The main entrypoint `gamnit` provides low-level abstractions over some services of OpenGL ES 2.0, like textures, shaders and programs. +* The main entrypoint [`gamnit`](gamnit) provides low-level abstractions over some services of OpenGL ES 2.0, like [textures](gamnit::Texture), [shaders](gamnit::Shader) and programs. It defines the basic methods to implement in order to obtain a working game: `App::frame_core` to update the screen and `App::accept_event` to receive user inputs. -* `flat` provides an easy to use API for 2D games based on sprites. +* [`flat`](gamnit>flat>) provides an easy to use API for 2D games based on sprites. Clients of this API redefine `App::update` to update the game logic and fill lists of sprites with objects to draw. `App::sprites` lists the sprites of the game world, they are drawn form the point of view of the `world_camera`. - This camera can be moved around in the world by updating the x and y of its `position`, + This [camera](gamnit::Camera) can be moved around in the world by updating the x and y of its `position`, and the zoom can easily be set using `reset_depth(desired_world_units_on_y)` or the `z` of its `position` `App::ui_sprites` lists the UI sprites drawn from the point of view of `ui_camera`. @@ -45,7 +111,7 @@ _gamnit_ is modular, different services of the framework are available through d However to support screens with different DPI, it is recommended to standardize the display size using `reset_depth(height_of_reference_display)`. -* `depth` defines an API for 3D games based on instances of `Actor`. +* [`depth`](gamnit>depth>) defines an API for 3D games based on instances of `Actor`. This framework is build upon `flat`, see the doc of this submodule first (just above). As such, clients should still implement `update` with the game logic, and fill `ui_sprites` for UI elements. @@ -58,6 +124,398 @@ _gamnit_ is modular, different services of the framework are available through d * `keys` provides `app.pressed_keys` keeping track of the currently pressed keys. -* `model_parsers` provides services to read and parse models from the asset folder. +* [`model_parsers`](gamnit>model_parsers>) provides services to read and parse models from the [asset](gamnit>virtual_gamepad>assets>) folder. + +* [`network`](gamnit>network>) provides a simple communication framework for multiplayer client/server games. + + * `android19` - Variation using features from Android API 19 + +* `bmfont` - Parse Angel Code BMFont format and draw text + +* `camera_control` - Simple camera control for user, as the method `accept_scroll_and_zoom` + +* `camera_control_android` - Two fingers camera manipulation, pinch to zoom and slide to scroll + +* `camera_control_linux` - Mouse wheel and middle mouse button to control camera + +* `cameras` - Camera services producing Model-View-Projection matrices + +* `cameras_cache` - Cache the `Matrix` produced by `Camera::mvp_matrix` + +* `cardboard` - Update the orientation of `world_camera` at each frame using the head position given by `android::cardboard` + +* `client` - Client-side network services for games and such + +* `common` - Services common to the `client` and `server` modules + +* `depth` - Framework for 3D games in Nit + +* `depth_core` - Base entities of the depth 3D game framework + +* `display` - Abstract display services + +* `display_android` - Gamnit display implementation for Android + +* `display_ios` - Gamnit display implementation for iOS + +* `display_linux` - Gamnit display implementation for GNU/Linux using `egl`, `sdl` and `x11` + +* `dynamic_resolution` - Virtual screen with a resolution independent from the real screen + +* `egl` - Use of EGL to implement Gamnit on GNU/Linux and Android + +* `flat` - Simple API for 2D games, built around `Sprite` and `App::update` + +* `flat_core` - Core services for the `flat` API for 2D games + +* `font` - Abstract font drawing services, implemented by `bmfont` and `tileset` + +* `gamnit` - Game and multimedia framework for Nit + +* `gamnit_android` - Support services for Gamnit on Android + +* `gamnit_ios` - Support services for gamnit on iOS + +* `gamnit_linux` - Support services for Gamnit on GNU/Linux + +* `input_ios` - Gamnit event support for iOS + +* `keys` - Simple service keeping track of which keys are currently pressed + +* `landscape` - Lock the application in the landscape orientation + +* `limit_fps` - Frame-rate control for applications + +* `model_dimensions` - Dimensions related services for `Model` and `Mesh` + +* `model_parser_base` - Services to parse models from a text description + +* `more_lights` - More implementations of `Light` + +* `more_materials` - Various material implementations + +* `more_meshes` - More simple geometric meshes + +* `more_models` - Services to load models from the assets folder + +* `mtl` - Services to parse .mtl material files + +* `network` - Easy client/server logic for games and simple distributed applications + +* `obj` - Services to parse .obj geometry files + +* `particles` - Particle effects + +* `portrait` - Lock the application in the portrait orientation + +* `programs` - Services for graphical programs with shaders, attributes and uniforms + +* `selection` - Select `Actor` from a screen coordinate + +* `server` - Server-side network services for games and such + +* `shadow` - Shadow mapping using a depth texture + +* `stereoscopic_view` - Refine `EulerCamera` and `App::frame_core_draw` to get a stereoscopic view + +* `texture_atlas_parser` - Tool to parse XML texture atlas and generated Nit code to access subtextures + +* `textures` - Load textures, create subtextures and manage their life-cycle + +* `tileset` - Support for `TileSet`, `TileSetFont` and drawing text with `TextSprites` + +* `virtual_gamepad` - Virtual gamepad mapped to keyboard keys for quick and dirty mobile support + +* `virtual_gamepad_spritesheet` + +* `vr` - VR support for gamnit depth, for Android only + +## `network` + +> Both `gamnit::client` and `gamnit::server` can be used separately or +> together by importing `gamnit::network`. +> Use both modules to create an program that discover local servers +> or create one if none is found: + +~~~ +redef fun handshake_app_name do return "network_test" + +# Discover local servers +var servers = discover_local_servers +if servers.not_empty then + # Try to connect to the first local server + var server_info = servers.first + var server = new RemoteServer(server_info) + + if not server.connect then + print_error "Failed to connect to {server_info.address}:{server_info.port}" + else if not server.handshake then + print_error "Failed handshake with {server_info.address}:{server_info.port}" + else + # Connected! + print "Connected to {server_info.address}:{server_info.port}" + + # Write something and close connection + server.writer.serialize "hello server" + server.socket.as(not null).close + end +else + # Create a local server + var connect_port = 33729 + print "Launching server: connect on {connect_port}, discovery on {discovery_port}" + var server = new Server(connect_port) + + # Don't loop if testing + if "NIT_TESTING".environ == "true" then exit 0 + + loop + # Respond to discovery requests + server.answer_discovery_requests + + # Accept new clients + var new_clients = server.accept_clients + for client in new_clients do + # Read something and close connection + assert client.reader.deserialize == "hello server" + client.socket.close + end + end +end +~~~ + +## `android19` + +> Add support for `TextureAsset::premultiply_alpha = false` on Android. + +## `flat` + +> Client programs should implement `App::update` to execute game logic and +> add instances of `Sprite` to `App::sprites` and `App::ui_sprites`. +> At each frame, all sprites are drawn to the screen. + +This system relies on two cameras `App::world_camera` and `App::ui_camera`. + +* `App::world_camera` applies a perspective effect to draw the game world. + This camera is designed to be moved around to see the world as well as to + zoom in and out. It is used to position the sprites in `App::sprites`. + +* `App::ui_camera` is a simple orthogonal camera to display UI objects. + This camera should mostly be still, it can still move for chock effects + and the like. It can be used to standardize the size of the UI across + devices. It is used to position the sprites in `App::ui_sprites`. + +See the sample game at `contrib/asteronits/` and the basic project template +at `lib/gamnit/examples/template/`. + +## `keys` + +> This service revolves around `app.pressed_keys`, a `Set` of the names of currently pressed keys. +> As a `Set`, `app.pressed_keys` can be iterated and queried with `has`. + +Limitations: The keys names are platform dependent. + +~~~nitish +redef class App + redef fun accept_event(event) + do + # First, pass the event to `super`, `pressed_keys` must see all + # events but it doesn't intercept any of them. + if super then return true + return false + end + + redef fun update(dt) + do + for key in pressed_keys do + if k == "left" or k == "a" then + # Act on key pressed down + print "left or a is pressed down" + end + end + end +end +~~~ + +## `display_android` + +> Gamnit apps on Android require OpenGL ES 3.0 because, even if it uses only +> the OpenGL ES 2.0 API, the default shaders have more than 8 vertex attributes. +> OpenGL ES 3.0 ensures at least 8 vertex attributes, while 2.0 ensures only 4. + +This module relies on `android::native_app_glue` and the Android NDK. + +## `cardboard` + +> This module is Android specific. + +## `client` + +> The following code implements a client to connect to a local server and +> briefly exchange with it. + +~~~ +redef fun handshake_app_name do return "nitwork_test" +redef fun handshake_app_version do return "1.0" + +# Prepare connection with remote server +var config = new RemoteServerConfig("localhost", 4444) +var server = new RemoteServer(config) + +# Try to connect +if not server.connect then return + +# Make sure the server is compatible +if not server.handshake then return + +# Connection up! communicate +server.writer.serialize "hello server" +print server.reader.deserialize.as(Object) + +# Done, close socket +server.socket.close +~~~ + +## `server` + +> The following code creates a server that continuously listen for new clients, +> and exchange with them briefly before disconnecting. + +~~~nitish +redef fun handshake_app_name do return "nitwork_test" +redef fun handshake_app_version do return "1.0" + +Open a server on port 4444 +var server = new Server(4444) + +loop + # Accept new clients + var new_clients = server.accept_clients + for client in new_clients do + # A client is connected, communicate! + print "" + print client.reader.deserialize.as(Object) + client.writer.serialize "Goodbye client" + + # Done, close socket + client.socket.close + end + + # `accept_clients` in non-blocking, + # sleep before tying again, or do something else. + 0.5.sleep + printn "." +end +~~~ + +## `particles` + +> Particles are managed by instances of `ParticleSystem` that +> are configured for a specific kind of particle. +> For instance, a particle system can be created for a max of 100 particles, +> with a smoke effect and a precise texture, as in: + +~~~ +var smoke = new ParticleSystem(100, app.smoke_program, new Texture("smoke.png")) +~~~ + +The system must be registered in `app.particle_systems` to be drawn on screen. + +Particles are added to a system with their configuration, as in: + +~~~ +var position = new Point3d[Float](0.0, 0.0, 0.0) +var scale = 2.0 +var time_to_live = 1.0 # in seconds +smoke.add(position, scale, time_to_live) +~~~ + +## `selection` + +> The two main services are `App::visible_at` and ; App::visible_in_center`. + +This is implemented with simple pixel picking. +This algorithm draws each actor in a unique color to the display buffer, +using the color as an ID to detect which actor is visible at each pixel. + +It is implemented at the level of the material, +so it can be applied to any _gamnit_ programs. +However it is not optimal performance wise, +so client programs should implement a more efficient algorithm. + +By default, the actors are drawn as opaque objects. +This behavior can be refined, as does `TexturedMaterial` to use its +`diffuse_texture` for partial opacity. + +## `dynamic_resolution` + +> The attribute `dynamic_resolution_ratio` sets the size of the virtual screen +> in relation to the real screen. See its documentation for more information. + +Reference implementation: +https://software.intel.com/en-us/articles/dynamic-resolution-rendering-on-opengl-es-2 + +## `virtual_gamepad` + +> For Android, the texture is automatically added to the APK when this +> module is imported. However, due to a limitation of the _app.nit_ +> framework on desktop OS, the texture must be copied manually to the assets +> folder at `assets/images`, the texture is available at, from the repo root, +> `lib/gamnit/virtual_gamepad/assets/images`. + +The texture was created by kenney.nl and modified by Alexis Laferrière. +It is published under CC0 and can be used and modified without attribution. + +~~~ +redef class App + redef fun create_scene + do + super + + # Create the virtual gamepad + var gamepad = new VirtualGamepad + + # Configure it as needed + gamepad.add_dpad(["w","a","s","d"]) + gamepad.add_button("x", gamepad_spritesheet.x) + gamepad.add_button("space", gamepad_spritesheet.star) + + # Assign it as the active gamepad + self.gamepad = gamepad + end + + fun setup_play_ui + do + # Show the virtual gamepad + var gamepad = self.gamepad + if gamepad != null then gamepad.visible = true + end +end +~~~ + +## `bmfont` + +> The BMFont format supports packed textures, varying advance per character and +> even kernings. It can be generated with a number of tools, inluding: + +* BMFont, free software Windows app, http://www.angelcode.com/products/bmfont/ +* Littera, a web app, http://kvazars.com/littera/ + +Format reference: http://www.angelcode.com/products/bmfont/doc/file_format.html + +## `shadow` + +> The default light does not cast any shadows. It can be changed to a +> `ParallelLight` in client games to cast sun-like shadows: + +~~~ +import more_lights + +var sun = new ParallelLight +sun.pitch = 0.25*pi +sun.yaw = 0.25*pi +app.light = sun +~~~ + +## Authors -* `network` provides a simple communication framework for multiplayer client/server games. +This project is maintained by **Alexis Laferrière **. diff --git a/lib/geometry/README.docdown.md b/lib/geometry/README.docdown.md new file mode 100644 index 0000000000..ccf41a6ff4 --- /dev/null +++ b/lib/geometry/README.docdown.md @@ -0,0 +1,90 @@ +# `geometry` - Basic geometry data structures and services + +[[toc: geometry]] + +[[features: geometry | mentities: geometry::geometry;geometry::angles;geometry::points_and_lines;geometry::boxes;geometry::quadtree;geometry::polygon]] + +## Points and Lines + +[[uml: geometry | format: svg, mentities: geometry::geometry;geometry::angles;geometry::points_and_lines;geometry::boxes;geometry::quadtree;geometry::polygon]] + +The very basics of [[geometry]] needs, for two and three-dimensional space. + +## Boxes and detection collision + +[[geometry::Box | text: Boxes]] module introduces Bounding [[geometry::boxes]] for Points and Lines and services to detect collision or inclusion between boxes. +It means a simple and fast way to test collision but not really accurate since it uses bounding boxes. + +### [[sign: geometry::Box]] + +> [[doc: geometry::Box]] + +### [[sign: geometry::Box3d]] + +> [[doc: geometry::Box3d]] + +### [[sign: geometry::BoxedArray]] + +> [[doc: geometry::BoxedArray]] + +## [[sign: geometry::quadtree]] + +> [[doc: geometry::quadtree]] + +A [[geometry::QuadTree]] is a tree data structure in which each internal node has exactly four children +They're most often used to partition two-dimensional space by recursively subdividing +it into four quadrants or regions. + +* They decompose space into adaptable cells +* Each cell has a maximum capacity. When maximum is reached, the cell splits. + +Quadtrees are using [[geometry::Boxed]] objects to determine their distribution in the 2D space. + +This API provides two different types of Quadtree : Static and Dynamic (respectively [[geometry::SQuadTree | text: `SQuadTree`]] and [[geometry::DQuadTree | text: `DQuadTree`]]). + +* Static: When you create the QuadTree, you need to specify the region that it will cover + +* Dynamic: You just need to fill the quadtree with objects, and when the threshold is reached, + it will automatically divide the current region, depending on the distribution of objects already in the region. + +### [[sign: geometry::SQuadTree]] + +> [[doc: geometry::SQuadTree]] + +### [[sign: geometry::DQuadTree]] + +> [[doc: geometry::DQuadTree]] + +## Polygons + +Some basic [[geometry::Polygon | text: polygon]] services. + +This module contains interesting algorithms for [[geometry::ConvexPolygon | text: `ConvexPolygon`]]only at the moment. A Convex [[geometry::polygon]] can be defined as follow : + +* All its interior angles are less than 180°. this means that all the vertices of the polygon + will [[geometry::Point | text: point]] outwards, away from the interior of the shape. + +* Every point on every [[geometry::Line | text: line]] segment between two points inside or on the boundary of the polygon + remains inside or on the boundary. + +* The polygon is entirely contained in a closed half-plane defined by each of its edges. + +* For each edge, the interior points are all on the same side of the line that the edge defines. + +* The angle at each vertex contains all other vertices in its edges and interior. + +A polygon which is not convex is called concave. Convex polygon are used because most +geometric problems are simpler and faster on convex objects than on non-convex ones. + +Services provided : + +* Point in convex polygon +* Intersection of convex polygon +* Convex hull of a set of points +* Triangulation of polygon + +[[features: geometry | mentities: geometry::Polygon;geometry::ConvexPolygon;geometry::Line;geometry::ILine3d;geometry::Line3d;geometry::Point;geometry::BoxedCollection;geometry::ILine;geometry::IPoint3d;geometry::Boxed3d;geometry::DQuadTree;geometry::BoxedArray;geometry::Point3d;geometry::IPoint;geometry::APolygon;geometry::Box3d;geometry::Box;geometry::Boxed;geometry::SQuadTree;geometry::QuadTree]] + +## Authors + +This project is maintained by [[ini-maintainer: geometry]]. diff --git a/lib/geometry/README.md b/lib/geometry/README.md index 3e20f59775..e8f54b64d9 100644 --- a/lib/geometry/README.md +++ b/lib/geometry/README.md @@ -1,43 +1,108 @@ -Basic geometry data structures and services. +# `geometry` - Basic geometry data structures and services -# Points and Lines +* [Points and Lines](#Points-and-Lines) -The very basics of geometry needs, for two and three-dimensional space. +* [Boxes and detection collision](#Boxes-and-detection-collision) -# Boxes and detection collision +* [`Box`](#`Box`) -Boxes module introduces Bounding boxes for Points and Lines and services to detect collision or inclusion between boxes. +* [`Box3d`](#`Box3d`) + +* [`BoxedArray`](#`BoxedArray`) + +* [`quadtree`](#`quadtree`) + +* [`SQuadTree`](#`SQuadTree`) + +* [`DQuadTree`](#`DQuadTree`) + +* [Polygons](#Polygons) + +* [Authors](#Authors) + +* `angles` - Angle related service using `Float` to represent an angle in radians + +* `boxes` - Provides interfaces and classes to represent basic geometry needs. + +* `geometry` - Provides interfaces and classes to represent basic geometry needs. + +* `points_and_lines` - Interfaces and classes to represent basic geometry needs. + +* `polygon` - Convex Polygons manipulations + +* `quadtree` - QuadTree API mostly used for 2 dimensional collision detection + +## Points and Lines + +![Diagram for `geometry`](uml-geometry.svg) + +The very basics of [geometry](geometry) needs, for two and three-dimensional space. + +## Boxes and detection collision + +[Boxes](geometry::Box) module introduces Bounding [boxes](geometry::boxes) for Points and Lines and services to detect collision or inclusion between boxes. It means a simple and fast way to test collision but not really accurate since it uses bounding boxes. -# Quadtrees +### `Box` + +> This class offers many constructors specialized for different usage. They are +> named according to the order of their arguments. + +### `Box3d` + +> This class offers many constructors specialized for different usage. They are +> named according to the order of their arguments. + +### `BoxedArray` -A QuadTree is a tree data structure in which each internal node has exactly four children +> Linear performances for searching, but really fast creation and filling. + +## `quadtree` + +> The QuadTree data structure partition a 2D space by recursively +> subdividing it into 4 regions when its capacity is reached. +> This module introduces 2 main implementation of the structure, +> a static and a dynamic QuadTree. + +A [QuadTree](geometry::QuadTree) is a tree data structure in which each internal node has exactly four children They're most often used to partition two-dimensional space by recursively subdividing it into four quadrants or regions. * They decompose space into adaptable cells * Each cell has a maximum capacity. When maximum is reached, the cell splits. -Quadtrees are using Boxed objects to determine their distribution in the 2D space. +Quadtrees are using [Boxed](geometry::Boxed) objects to determine their distribution in the 2D space. -This API provides two different types of Quadtree : Static and Dynamic (respectively `SQuadTree` and `DQuadTree`). +This API provides two different types of Quadtree : Static and Dynamic (respectively [`SQuadTree`](geometry::SQuadTree) and [`DQuadTree`](geometry::DQuadTree)). * Static: When you create the QuadTree, you need to specify the region that it will cover * Dynamic: You just need to fill the quadtree with objects, and when the threshold is reached, -it will automatically divide the current region, depending on the distribution of objects already in the region. + it will automatically divide the current region, depending on the distribution of objects already in the region. + +### `SQuadTree` + +> You need to specify a zone when creating the quadtree, +> which will be the zone corresponding to the root node. +> Each subdivision cut the space in 4 equal regions from +> the center of the parent node. + +### `DQuadTree` -# Polygons +> The center of the parent node is determined by the average +> values of the data it contains when `item_limit` is reached. -Some basic polygon services. +## Polygons -This module contains interesting algorithms for `ConvexPolygon`only at the moment. A Convex polygon can be defined as follow : +Some basic [polygon](geometry::Polygon) services. + +This module contains interesting algorithms for [`ConvexPolygon`](geometry::ConvexPolygon)only at the moment. A Convex [polygon](geometry::polygon) can be defined as follow : * All its interior angles are less than 180°. this means that all the vertices of the polygon -will point outwards, away from the interior of the shape. + will [point](geometry::Point) outwards, away from the interior of the shape. -* Every point on every line segment between two points inside or on the boundary of the polygon -remains inside or on the boundary. +* Every point on every [line](geometry::Line) segment between two points inside or on the boundary of the polygon + remains inside or on the boundary. * The polygon is entirely contained in a closed half-plane defined by each of its edges. @@ -51,6 +116,53 @@ geometric problems are simpler and faster on convex objects than on non-convex o Services provided : * Point in convex polygon + * Intersection of convex polygon + * Convex hull of a set of points + * Triangulation of polygon + +* `APolygon` - Abstraction of a Polygon + +* `Box` - A 2d bounded object and an implementation of `Boxed` + +* `Box3d` - A 3d bounded object and an implementation of Boxed + +* `Boxed` - An 2d abstract bounded object + +* `Boxed3d` - An 3d abstract bounded object + +* `BoxedArray` - `BoxedCollection` implemented by an array + +* `BoxedCollection` - Base for all data structures containing multiple Boxed Objects + +* `ConvexPolygon` - Convex Polygon class + +* `DQuadTree` - A dynamic implementation of the quadtree data structure + +* `ILine` - Abstract 2D line segment between two ordered points + +* `ILine3d` - Abstract 3D line segment between two ordered points + +* `IPoint` - Abstract 2d point, strongly linked to its implementation `Point` + +* `IPoint3d` - Abstract 3d point, strongly linked to its implementation `Point3d` + +* `Line` - 2D line segment between two ordered points + +* `Line3d` - 3D line segment between two ordered points + +* `Point` - 2D point with `x` and `z` + +* `Point3d` - 3D point with `x`, `y` and `z` + +* `Polygon` - A simple polygon + +* `QuadTree` - Abstract QuadTree implementing the basic functions and data + +* `SQuadTree` - Static implementation of the quadtree structure + +## Authors + +This project is maintained by **Romain Chanoir **. diff --git a/lib/github/README.docdown.md b/lib/github/README.docdown.md new file mode 100644 index 0000000000..152cb4d6f6 --- /dev/null +++ b/lib/github/README.docdown.md @@ -0,0 +1,393 @@ +# `github` - Nit wrapper for Github API + +[[toc: github]] + +This module provides a Nit object oriented interface to access the [[github::github | text: Github]] [[github::api]]. + +[[features: github | mentities: github::github;github::cache;github::github_curl;github::wallet;github::hooks;github::events;github::loader;github::api]] + +## Getting Started + +These instructions will get you a copy of the project up and running on your local machine. + +### Dependencies + +This project requires the following packages: + +[[parents: github]] + +### Run `loader` + +[[uml: github | format: svg, mentities: github::github;github::cache;github::github_curl;github::wallet;github::hooks;github::events;github::loader;github::api]] + +Compile `loader` with the following command: + +[[main-compile: github::loader]] + +Then run it with: + +[[main-run: github::loader]] + +Options: + +[[main-opts: github::loader]] + +## Accessing the API + +### `GithubAPI` - # Client to Github API + +To access the API you need an instance of a [[github::GithubAPI | text: `GithubAPI`]] client. + +~~~ +import github + +# Get Github authentification token. +var token = get_github_oauth +assert not token.is_empty + +# Init the client. +var api = new GithubAPI(token) +~~~ + +The API client allows you to get [[github | text: Github]] API entities. + +~~~ +import github + +var api = new GithubAPI(get_github_oauth) +var repo = api.load_repo("nitlang/nit") +assert repo != null +assert repo.name == "nit" + +var user = api.load_user("Morriar") +assert user != null +assert user.login == "Morriar" +~~~ + +[[features: github | mentities: github::RepoRepo;github::IssueRepo;github::CommitRepo;github::BranchRepo;github::IssueEventRepo;github::PullRequestRepo;github::IssueCommentRepo;github::LoaderJobRepo;github::CommitCommentEvent;github::GithubDeserializer;github::GithubFile;github::ForkEvent;github::MemberEvent;github::GitUser;github::Label;github::IssueCommentEvent;github::DeleteEvent;github::RenameAction;github::LoaderJob;github::PullRequestReviewCommentEvent;github::GithubCurl;github::RepoEntity;github::Branch;github::CommitComment;github::CreateEvent;github::PushEvent;github::GithubError;github::DeploymentStatusEvent;github::GitCommit;github::ContributorStats;github::PullRef;github::IssueComment;github::StatusEvent;github::Repo;github::User;github::PullRequestEvent;github::HookListener;github::GithubWallet;github::IssuesEvent;github::DeploymentEvent;github::ReviewComment;github::GithubEntity;github::Commit;github::IssueEvent;github::GithubEvent;github::Milestone;github::Comment;github::Loader;github::PullRequest;github::Issue;github::GithubAPI;github::LoaderConfig]] + +### Authentification + +Token can also be recovered from user config with `get_github_oauth`. + +### `get_github_oauth` - # Gets the Github token from `git` configuration + +Return the value of `git config --get github.oauthtoken` +or `""` if no key exists. + +### Retrieving user data + +### `load_user` - # Get the Github user with `login` + +Loads the [[github::User | text: `User`]] from the API or returns `null` if the user cannot be found. + +~~~ +import github + +var api = new GithubAPI(get_github_oauth) +var user = api.load_user("Morriar") +print user or else "null" +assert user.login == "Morriar" +~~~ + +### `User` - # A Github user + +Provides access to [Github user data](https://developer.github.com/v3/users/). +Should be accessed from `GithubAPI::load_user`. + +* `api$User$SELF` - # Type of this instance, automatically specialized in every class + +* `_avatar_url` - # Avatar image url for this user. + +* `_blog` - # User public blog if any. + +* `_email` - # User public email if any. + +* `_login` - # Github login. + +* `_name` - # User public name if any. + +* `avatar_url` - # Avatar image url for this user. + +* `avatar_url=` - # Avatar image url for this user. + +* `blog` - # User public blog if any. + +* `blog=` - # User public blog if any. + +* `api$User$core_serialize_to` - # Actual serialization of `self` to `serializer` + +* `email` - # User public email if any. + +* `email=` - # User public email if any. + +* `api$User$from_deserializer` - # Create an instance of this class from the `deserializer` + +* `api$User$init` + +* `login` - # Github login. + +* `login=` - # Github login. + +* `name` - # User public name if any. + +* `name=` - # User public name if any. + +### Retrieving repo data + +### `load_repo` - # Get the Github repo with `full_name`. + +Loads the [[github::Repo | text: `Repo`]] from the API or returns `null` if the repo cannot be found. + +~~~ +import github + +var api = new GithubAPI(get_github_oauth) +var repo = api.load_repo("nitlang/nit") +assert repo.name == "nit" +assert repo.owner.login == "nitlang" +assert repo.default_branch == "master" +~~~ + +### `Repo` - # A Github repository. + +Provides access to [Github repo data](https://developer.github.com/v3/repos/). +Should be accessed from `GithubAPI::load_repo`. + +* `api$Repo$SELF` - # Type of this instance, automatically specialized in every class + +* `_default_branch` - # Repo default [[github::Branch | text: branch]] name. + +* `_full_name` - # Repo full name on Github. + +* `_mongo_id` + +* `_name` - # Repo short name on Github. + +* `_owner` - # Get the repo owner. + +* `loader$Repo$core_serialize_to` - # Actual serialization of `self` to `serializer` + +* `api$Repo$core_serialize_to` - # Actual serialization of `self` to `serializer` + +* `default_branch` - # Repo default branch name. + +* `default_branch=` - # Repo default branch name. + +* `api$Repo$from_deserializer` - # Create an instance of this class from the `deserializer` + +* `loader$Repo$from_deserializer` - # Create an instance of this class from the `deserializer` + +* `full_name` - # Repo full name on Github. + +* `full_name=` - # Repo full name on Github. + +* `api$Repo$init` + +* `mongo_id` + +* `mongo_id=` + +* `name` - # Repo short name on Github. + +* `name=` - # Repo short name on Github. + +* `owner` - # Get the repo owner. + +* `owner=` - # Get the repo owner. + +### Other data + +* `api$Branch` - # A Github branch. + +* `api$Comment` - # A Github [[github::Comment | text: comment]] + +* `api$Commit` - # A Github [[github::Commit | text: commit]]. + +* `api$CommitComment` - # A comment made on a commit. + +* `api$ContributorStats` - # Should be accessed from `Repo::contrib_stats`. + +* `api$Deserializer` - # Abstract deserialization service + +* `api$GitCommit` - # A Git Commit representation + +* `api$GitUser` - # Git user authoring data + +* `api$GithubAPI` - # Client to Github API + +* `api$GithubDeserializer` - # JsonDeserializer specific for Github objects. + +* `api$GithubEntity` - # Something returned by the Github API. + +* `api$GithubFile` - # A Github file representation. + +* `api$ISODate` - # Make ISO Datew serilizable + +* `api$Issue` - # A Github issue. + +* `api$IssueComment` - # Comments made on Github issue and pull request pages. + +* `api$IssueEvent` - # An event that occurs on a Github [[github::Issue | text: `Issue`]]. + +* `api$Label` - # A Github [[github::Label | text: label]]. + +* `api$Milestone` - # A Github [[github::Milestone | text: milestone]]. + +* `api$PullRef` - # A pull request reference (used for head and base). + +* `api$PullRequest` - # A Github pull request. + +* `api$RenameAction` - # A rename action maintains the name before and after a renaming action. + +* `api$Repo` - # A Github repository. + +* `api$ReviewComment` - # Comments made on Github pull request diffs. + +* `api$User` - # A Github user + +### Advanced uses + +#### Caching + +#### Custom requests + +### `get` - # Execute a GET request on Github API. + +This method returns raw json data. +See other `load_*` methods to use more expressive types. + +~~~ +import github + +var api = new GithubAPI(get_github_oauth) +var obj = api.get("/repos/nitlang/nit") +assert obj isa JsonObject +assert obj["name"] == "nit" +~~~ + +Returns `null` in case of `error`. + +~~~ +import github + +var api = new GithubAPI(get_github_oauth) +var obj = api.get("/foo/bar/baz") +assert obj == null +assert api.was_error +var err = api.last_error +assert err isa GithubError +assert err.name == "GithubAPIError" +assert err.message == "Not Found" +~~~ + +#### Change the user agent + +### `user_agent` - # User agent used for HTTP requests. + +Default is `nit_github_api`. + +See + +#### Debugging + +### `verbose_lvl` - # Verbosity level. + +* `0`: only errors (default) +* `1`: verbose + +#### Using with GitLab + +If URL scheme of GitLab API follows the one of Github API, it may be possible to +configure this wrapper to use a custom URL. + +### `api_url` - # Github API base url. + +Default is `https://api.github.com` and should not be changed. + +## Creating hooks + +Using this API you can create Github hooks able to respond to actions performed +on a repository. + +### `hooks` - # Github hook event listening with `nitcorn`. + +Usage: + +~~~ +import github::hooks + +# A simple hook listener that print received events in stdout. +class LogHookListener + super HookListener + + # Use double dispatch to implement log behavior. + redef fun apply_event(event) do event.log_event(self) +end + +redef class GithubEvent + # Log this event. + # + # Do nothing by default. + fun log_event(l: LogHookListener) do end +end + +redef class CommitCommentEvent + + redef fun log_event(l) do + print "new comment on commit {comment.commit_id}" + end +end + +var api = new GithubAPI(get_github_oauth) +var listener = new LogHookListener(api, "127.0.0.1", 8080) +# listener.listen # uncomment to start listening +~~~ + +## Dealing with events + +GithubAPI can trigger different events depending on the hook configuration. + +### `GithubEvent` - # Github event stub. + +* `events$CommitCommentEvent` - # Triggered when a commit comment is created. + +* `events$CreateEvent` - # Triggered when a repository, branch, or tag is created. + +* `events$DeleteEvent` - # Triggered when a branch or a tag is deleted. + +* `events$DeploymentEvent` - # Triggered when a new snapshot is deployed. + +* `events$DeploymentStatusEvent` - # Triggered when a deployement's status changes. + +* `events$Deserializer` - # Abstract deserialization service + +* `events$ForkEvent` - # Triggered when a user forks a repository. + +* `events$GithubDeserializer` - # JsonDeserializer specific for Github objects. + +* `events$GithubEvent` - # Github event stub. + +* `events$IssueCommentEvent` - # Triggered when an issue comment is created. + +* `events$IssuesEvent` - # Triggered when an event occurs on an issue. + +* `events$MemberEvent` - # Triggered when a user is added as a collaborator to a repository. + +* `events$PullRequestEvent` - # Triggered when an event occurs on a pull request. + +* `events$PullRequestReviewCommentEvent` - # Triggered when a comment is created on a pull request diff. + +* `events$PushEvent` - # Triggered when a repository branch is pushed to. + +* `events$StatusEvent` - # Triggered when the status of a Git commit changes. + +## Running the tests + +Run the nitunit automated tests with the following command: + +[[testing: github]] + +## Authors + +This project is maintained by [[ini-maintainer: github]]. diff --git a/lib/github/README.md b/lib/github/README.md index 18814ee4da..e151d44b57 100644 --- a/lib/github/README.md +++ b/lib/github/README.md @@ -1,71 +1,506 @@ -# Nit wrapper for Github API - -This module provides a Nit object oriented interface to access the Github api. +# `github` - Nit wrapper for Github API + +* [Getting Started](#Getting-Started) +* [Dependencies](#Dependencies) +* [Run `loader`](#Run-`loader`) +* [Accessing the API](#Accessing-the-API) +* [`GithubAPI` - # Client to Github API](#`GithubAPI`---#-Client-to-Github-API) +* [Authentification](#Authentification) +* [`get_github_oauth` - # Gets the Github token from `git` configuration](#`get_github_oauth`---#-Gets-the-Github-token-from-`git`-configuration) +* [Retrieving user data](#Retrieving-user-data) +* [`load_user` - # Get the Github user with `login`](#`load_user`---#-Get-the-Github-user-with-`login`) +* [`User` - # A Github user](#`User`---#-A-Github-user) +* [Retrieving repo data](#Retrieving-repo-data) +* [`load_repo` - # Get the Github repo with `full_name`.](#`load_repo`---#-Get-the-Github-repo-with-`full_name`.) +* [`Repo` - # A Github repository.](#`Repo`---#-A-Github-repository.) +* [Other data](#Other-data) +* [Advanced uses](#Advanced-uses) +* [Caching](#Caching) +* [Custom requests](#Custom-requests) +* [`get` - # Execute a GET request on Github API.](#`get`---#-Execute-a-GET-request-on-Github-API.) +* [Change the user agent](#Change-the-user-agent) +* [`user_agent` - # User agent used for HTTP requests.](#`user_agent`---#-User-agent-used-for-HTTP-requests.) +* [Debugging](#Debugging) +* [`verbose_lvl` - # Verbosity level.](#`verbose_lvl`---#-Verbosity-level.) +* [Using with GitLab](#Using-with-GitLab) +* [`api_url` - # Github API base url.](#`api_url`---#-Github-API-base-url.) +* [Creating hooks](#Creating-hooks) +* [`hooks` - # Github hook event listening with `nitcorn`.](#`hooks`---#-Github-hook-event-listening-with-`nitcorn`.) +* [Dealing with events](#Dealing-with-events) +* [`GithubEvent` - # Github event stub.](#`GithubEvent`---#-Github-event-stub.) +* [Running the tests](#Running-the-tests) +* [Authors](#Authors) + +This module provides a Nit object oriented interface to access the [Github](github::github) [api](github::api). + +* `api` - Nit object oriented interface to [Github api](https://developer.github.com/v3/). +* `cache` - Enable caching on Github API accesses. +* `events` - Events are emitted by Github Hooks. +* `github` - Nit wrapper for Github API +* `github_curl` - Curl extention to access the Github API +* `hooks` - Github hook event listening with `nitcorn`. +* `loader` +* `wallet` - Github OAuth tokens management + +## Getting Started + +These instructions will get you a copy of the project up and running on your local machine. + +### Dependencies + +This project requires the following packages: + +* `console` - Defines some ANSI Terminal Control Escape Sequences. +* `curl` - Data transfer powered by the native curl library +* `json` - read and write JSON formatted text +* `nitcorn` - Lightweight framework for Web applications development +* `popcorn` - Web application framework for Nit + +### Run `loader` + +![Diagram for `github`](uml-github.svg) + +Compile `loader` with the following command: + +~~~bash +nitc ./loader.nit +~~~ + +Then run it with: + +~~~bash +loader +~~~ + +Options: + +~~~bash +* -h, -?, --help Show this help message. +* --config Path to config file. +* --db-host MongoDb host. +* --db-name MongoDb database name. +* --tokens Token list. +* --show-wallet Show wallet status. +* -v, --verbose Verbosity level. +* --no-colors Do not use colors in output. +* --show-jobs Show jobs status. +* --no-commits Do not load commits from default branch. +* --no-issues Do not load issues. +* --no-comments Do not load issue comments. +* --no-events Do not load issues events. +* --from Start loading issues from a number. +* --clear Clear job for given repo name. +~~~ ## Accessing the API -[[doc: GithubAPI]] +### `GithubAPI` - # Client to Github API + +To access the API you need an instance of a [`GithubAPI`](github::GithubAPI) client. + +~~~ +import github + +# Get Github authentification token. +var token = get_github_oauth +assert not token.is_empty + +# Init the client. +var api = new GithubAPI(token) +~~~ + +The API client allows you to get [Github](github) API entities. + +~~~ +import github + +var api = new GithubAPI(get_github_oauth) +var repo = api.load_repo("nitlang/nit") +assert repo != null +assert repo.name == "nit" + +var user = api.load_user("Morriar") +assert user != null +assert user.login == "Morriar" +~~~ + +* `Branch` - A Github branch. +* `BranchRepo` +* `Comment` - A Github comment +* `Commit` - A Github commit. +* `CommitComment` - A comment made on a commit. +* `CommitCommentEvent` - Triggered when a commit comment is created. +* `CommitRepo` +* `ContributorStats` - Should be accessed from `Repo::contrib_stats`. +* `CreateEvent` - Triggered when a repository, branch, or tag is created. +* `DeleteEvent` - Triggered when a branch or a tag is deleted. +* `DeploymentEvent` - Triggered when a new snapshot is deployed. +* `DeploymentStatusEvent` - Triggered when a deployement's status changes. +* `ForkEvent` - Triggered when a user forks a repository. +* `GitCommit` - A Git Commit representation +* `GitUser` - Git user authoring data +* `GithubAPI` - Client to Github API +* `GithubCurl` - Specific Curl that know hot to talk to the github API +* `GithubDeserializer` - JsonDeserializer specific for Github objects. +* `GithubEntity` - Something returned by the Github API. +* `GithubError` - An error thrown by the Github API. +* `GithubEvent` - Github event stub. +* `GithubFile` - A Github file representation. +* `GithubWallet` - Github OAuth tokens wallet +* `HookListener` - A nitcorn listener for Github hooks. +* `Issue` - A Github issue. +* `IssueComment` - Comments made on Github issue and pull request pages. +* `IssueCommentEvent` - Triggered when an issue comment is created. +* `IssueCommentRepo` +* `IssueEvent` - An event that occurs on a Github `Issue`. +* `IssueEventRepo` +* `IssueRepo` +* `IssuesEvent` - Triggered when an event occurs on an issue. +* `Label` - A Github label. +* `Loader` +* `LoaderConfig` - Loader configuration file +* `LoaderJob` - Loader status by repo +* `LoaderJobRepo` - Loader status repository +* `MemberEvent` - Triggered when a user is added as a collaborator to a repository. +* `Milestone` - A Github milestone. +* `PullRef` - A pull request reference (used for head and base). +* `PullRequest` - A Github pull request. +* `PullRequestEvent` - Triggered when an event occurs on a pull request. +* `PullRequestRepo` +* `PullRequestReviewCommentEvent` - Triggered when a comment is created on a pull request diff. +* `PushEvent` - Triggered when a repository branch is pushed to. +* `RenameAction` - A rename action maintains the name before and after a renaming action. +* `Repo` - A Github repository. +* `RepoEntity` +* `RepoRepo` +* `ReviewComment` - Comments made on Github pull request diffs. +* `StatusEvent` - Triggered when the status of a Git commit changes. +* `User` - A Github user ### Authentification -[[doc: auth]] - Token can also be recovered from user config with `get_github_oauth`. -[[doc: get_github_oauth]] +### `get_github_oauth` - # Gets the Github token from `git` configuration + +Return the value of `git config --get github.oauthtoken` +or `""` if no key exists. ### Retrieving user data -[[doc: load_user]] -[[doc: User]] -[[defs: User]] +### `load_user` - # Get the Github user with `login` + +Loads the [`User`](github::User) from the API or returns `null` if the user cannot be found. + +~~~ +import github + +var api = new GithubAPI(get_github_oauth) +var user = api.load_user("Morriar") +print user or else "null" +assert user.login == "Morriar" +~~~ + +### `User` - # A Github user + +Provides access to [Github user data](https://developer.github.com/v3/users/). +Should be accessed from `GithubAPI::load_user`. + +* `api$User$SELF` - # Type of this instance, automatically specialized in every class + +* `_avatar_url` - # Avatar image url for this user. + +* `_blog` - # User public blog if any. + +* `_email` - # User public email if any. + +* `_login` - # Github login. + +* `_name` - # User public name if any. + +* `avatar_url` - # Avatar image url for this user. + +* `avatar_url=` - # Avatar image url for this user. + +* `blog` - # User public blog if any. + +* `blog=` - # User public blog if any. + +* `api$User$core_serialize_to` - # Actual serialization of `self` to `serializer` + +* `email` - # User public email if any. + +* `email=` - # User public email if any. + +* `api$User$from_deserializer` - # Create an instance of this class from the `deserializer` + +* `api$User$init` + +* `login` - # Github login. + +* `login=` - # Github login. + +* `name` - # User public name if any. + +* `name=` - # User public name if any. ### Retrieving repo data -[[doc: load_repo]] -[[doc: Repo]] -[[defs: Repo]] +### `load_repo` - # Get the Github repo with `full_name`. + +Loads the [`Repo`](github::Repo) from the API or returns `null` if the repo cannot be found. + +~~~ +import github + +var api = new GithubAPI(get_github_oauth) +var repo = api.load_repo("nitlang/nit") +assert repo.name == "nit" +assert repo.owner.login == "nitlang" +assert repo.default_branch == "master" +~~~ + +### `Repo` - # A Github repository. + +Provides access to [Github repo data](https://developer.github.com/v3/repos/). +Should be accessed from `GithubAPI::load_repo`. + +* `api$Repo$SELF` - # Type of this instance, automatically specialized in every class + +* `_default_branch` - # Repo default [branch](github::Branch) name. + +* `_full_name` - # Repo full name on Github. + +* `_mongo_id` + +* `_name` - # Repo short name on Github. + +* `_owner` - # Get the repo owner. + +* `loader$Repo$core_serialize_to` - # Actual serialization of `self` to `serializer` + +* `api$Repo$core_serialize_to` - # Actual serialization of `self` to `serializer` + +* `default_branch` - # Repo default branch name. + +* `default_branch=` - # Repo default branch name. + +* `api$Repo$from_deserializer` - # Create an instance of this class from the `deserializer` + +* `loader$Repo$from_deserializer` - # Create an instance of this class from the `deserializer` + +* `full_name` - # Repo full name on Github. + +* `full_name=` - # Repo full name on Github. + +* `api$Repo$init` + +* `mongo_id` + +* `mongo_id=` + +* `name` - # Repo short name on Github. + +* `name=` - # Repo short name on Github. + +* `owner` - # Get the repo owner. + +* `owner=` - # Get the repo owner. ### Other data -[[defs: github::api]] +* `api$Branch` - # A Github branch. + +* `api$Comment` - # A Github [comment](github::Comment) + +* `api$Commit` - # A Github [commit](github::Commit). + +* `api$CommitComment` - # A comment made on a commit. + +* `api$ContributorStats` - # Should be accessed from `Repo::contrib_stats`. + +* `api$Deserializer` - # Abstract deserialization service + +* `api$GitCommit` - # A Git Commit representation + +* `api$GitUser` - # Git user authoring data + +* `api$GithubAPI` - # Client to Github API + +* `api$GithubDeserializer` - # JsonDeserializer specific for Github objects. + +* `api$GithubEntity` - # Something returned by the Github API. + +* `api$GithubFile` - # A Github file representation. + +* `api$ISODate` - # Make ISO Datew serilizable + +* `api$Issue` - # A Github issue. + +* `api$IssueComment` - # Comments made on Github issue and pull request pages. + +* `api$IssueEvent` - # An event that occurs on a Github [`Issue`](github::Issue). + +* `api$Label` - # A Github [label](github::Label). + +* `api$Milestone` - # A Github [milestone](github::Milestone). + +* `api$PullRef` - # A pull request reference (used for head and base). + +* `api$PullRequest` - # A Github pull request. + +* `api$RenameAction` - # A rename action maintains the name before and after a renaming action. + +* `api$Repo` - # A Github repository. + +* `api$ReviewComment` - # Comments made on Github pull request diffs. + +* `api$User` - # A Github user ### Advanced uses #### Caching -[[doc: cache]] - #### Custom requests -[[doc: github::GithubAPI::get]] +### `get` - # Execute a GET request on Github API. + +This method returns raw json data. +See other `load_*` methods to use more expressive types. + +~~~ +import github + +var api = new GithubAPI(get_github_oauth) +var obj = api.get("/repos/nitlang/nit") +assert obj isa JsonObject +assert obj["name"] == "nit" +~~~ + +Returns `null` in case of `error`. + +~~~ +import github + +var api = new GithubAPI(get_github_oauth) +var obj = api.get("/foo/bar/baz") +assert obj == null +assert api.was_error +var err = api.last_error +assert err isa GithubError +assert err.name == "GithubAPIError" +assert err.message == "Not Found" +~~~ #### Change the user agent -[[doc: github::GithubAPI::user_agent]] +### `user_agent` - # User agent used for HTTP requests. + +Default is `nit_github_api`. + +See #### Debugging -[[doc: verbose_lvl]] +### `verbose_lvl` - # Verbosity level. + +* `0`: only errors (default) +* `1`: verbose #### Using with GitLab If URL scheme of GitLab API follows the one of Github API, it may be possible to configure this wrapper to use a custom URL. -[[doc: api_url]] +### `api_url` - # Github API base url. + +Default is `https://api.github.com` and should not be changed. ## Creating hooks Using this API you can create Github hooks able to respond to actions performed on a repository. -[[doc: hooks]] +### `hooks` - # Github hook event listening with `nitcorn`. + +Usage: + +~~~ +import github::hooks + +# A simple hook listener that print received events in stdout. +class LogHookListener + super HookListener + + # Use double dispatch to implement log behavior. + redef fun apply_event(event) do event.log_event(self) +end + +redef class GithubEvent + # Log this event. + # + # Do nothing by default. + fun log_event(l: LogHookListener) do end +end + +redef class CommitCommentEvent + + redef fun log_event(l) do + print "new comment on commit {comment.commit_id}" + end +end + +var api = new GithubAPI(get_github_oauth) +var listener = new LogHookListener(api, "127.0.0.1", 8080) +# listener.listen # uncomment to start listening +~~~ ## Dealing with events GithubAPI can trigger different events depending on the hook configuration. -[[doc: GithubEvent]] +### `GithubEvent` - # Github event stub. + +* `events$CommitCommentEvent` - # Triggered when a commit comment is created. + +* `events$CreateEvent` - # Triggered when a repository, branch, or tag is created. + +* `events$DeleteEvent` - # Triggered when a branch or a tag is deleted. + +* `events$DeploymentEvent` - # Triggered when a new snapshot is deployed. + +* `events$DeploymentStatusEvent` - # Triggered when a deployement's status changes. + +* `events$Deserializer` - # Abstract deserialization service + +* `events$ForkEvent` - # Triggered when a user forks a repository. + +* `events$GithubDeserializer` - # JsonDeserializer specific for Github objects. + +* `events$GithubEvent` - # Github event stub. + +* `events$IssueCommentEvent` - # Triggered when an issue comment is created. + +* `events$IssuesEvent` - # Triggered when an event occurs on an issue. + +* `events$MemberEvent` - # Triggered when a user is added as a collaborator to a repository. + +* `events$PullRequestEvent` - # Triggered when an event occurs on a pull request. + +* `events$PullRequestReviewCommentEvent` - # Triggered when a comment is created on a pull request diff. + +* `events$PushEvent` - # Triggered when a repository branch is pushed to. + +* `events$StatusEvent` - # Triggered when the status of a Git commit changes. + +## Running the tests + +Run the nitunit automated tests with the following command: + +~~~bash +nitunit . +~~~ + +## Authors -[[defs: github::events]] +This project is maintained by **Alexandre Terrasa **. diff --git a/lib/ios/README.docdown.md b/lib/ios/README.docdown.md new file mode 100644 index 0000000000..55305df644 --- /dev/null +++ b/lib/ios/README.docdown.md @@ -0,0 +1,73 @@ +# `ios` - iOS support for _app.nit_ + +[[toc: ios]] + +[[features: ios | mentities: ios::platform;ios::ios;ios::http_request;ios::assets;ios::data_store;ios::app;ios::audio;ios::glkit;ios::uikit;ios::ui]] + +## Getting Started + +These instructions will get you a copy of the project up and running on your local machine. + +### Dependencies + +This project requires the following packages: + +[[parents: ios]] + +### Run `app` + +[[uml: ios | format: svg, mentities: ios::platform;ios::ios;ios::http_request;ios::assets;ios::data_store;ios::app;ios::audio;ios::glkit;ios::uikit;ios::ui]] + +Compile `app` with the following command: + +[[main-compile: ios::app]] + +Then run it with: + +[[main-run: ios::app]] + +## System configuration + +Configure your system for OS X by installing Xcode and brew. +Follow the instructions in `README.md` at the root of the repository. + +Optionally, install ios-sim to run compiled apps in the simulator: `brew install ios-sim` + +## Compile and run a simple application + +Let's use the `hello_ios` example in the folder `lib/ios/examples/`. + +Compile with: `nitc hello_world.nit` + +Run in the simulator with: `ios-sim hello_world.app` + +[[features: ios | mentities: ios::UIEvent;ios::UIApplication;ios::UITouch;ios::AppDelegate;ios::UITableViewStyle;ios::UITableViewDataSource;ios::UIWindow;ios::NSSet_UITouch;ios::UIScrollView;ios::UISwitch;ios::UITableViewCell;ios::NitViewController;ios::UILayoutConstraintAxis;ios::UITableViewDelegate;ios::UIButtonType;ios::UIButton;ios::UIControl;ios::UIViewController;ios::UIStackViewDistribution;ios::UITableView;ios::UITextField;ios::UIStackViewAlignment;ios::UILabel;ios::TableView;ios::UIStackView;ios::NitGLKView;ios::UIColor;ios::UIView]] + +## Sample portable applications + +See the calculator example at `examples/calculator` and the Tnitter client at `contrib/tnitter/` +for portable applications working on GNU/Linux, OS X, [[ios | text: iOS]] and Android. + +Example from `ios::hello_ios`: + +[[code: ios::hello_ios]] + +## Application icon + +To associate icons to your application, create the directory `ios/AppIcon.appiconset` and fill it with standard icons and `Contents.json`. +These files can be generated in a number of different ways: + +* Using the tool `svg_to_icons` packaged with the Nit repository at `contrib/inkscape_tools/bin/svg_to_icons`. + +* Using Xcode to assign images to each slot, create the folder and the file `Contents.json`. + +* Write or modify the file `Contents.json` manually. + It is in Json format and easily readable. + +## [[sign: ios::uikit]] + +> [[doc: ios::uikit]] + +## Authors + +This project is maintained by [[ini-maintainer: ios]]. diff --git a/lib/ios/README.md b/lib/ios/README.md index e4f7cfe2d7..2d51e55a2a 100644 --- a/lib/ios/README.md +++ b/lib/ios/README.md @@ -1,13 +1,80 @@ -iOS support for _app.nit_ +# `ios` - iOS support for _app.nit_ -# System configuration +* [Getting Started](#Getting-Started) + +* [Dependencies](#Dependencies) + +* [Run `app`](#Run-`app`) + +* [System configuration](#System-configuration) + +* [Compile and run a simple application](#Compile-and-run-a-simple-application) + +* [Sample portable applications](#Sample-portable-applications) + +* [Application icon](#Application-icon) + +* [`uikit`](#`uikit`) + +* [Authors](#Authors) + +* `app` - Basic structure for Nit apps on iOS + +* `assets` - Implementation of `app::assets` + +* `audio` - iOS implementation of `app::audio` using `AVAudioPlayer` + +* `data_store` - Implements `app::data_store` using `NSUserDefaults` + +* `glkit` - GLKit services to create an OpenGL ES context on iOS + +* `http_request` - Implementation of `app::http_request` for iOS + +* `ios` - iOS platform support + +* `platform` - Triggers compilation for the iOS platform + +* `ui` - Implementation of `app::ui` for iOS + +* `uikit` - File generated by objcwrapper + +## Getting Started + +These instructions will get you a copy of the project up and running on your local machine. + +### Dependencies + +This project requires the following packages: + +* `app` - _app.nit_, a framework for portable applications +* `cocoa` - Cocoa API, the development layer of OS X +* `core` - Nit common library of core classes and methods +* `json` - read and write JSON formatted text + +### Run `app` + +![Diagram for `ios`](uml-ios-2.svg) + +Compile `app` with the following command: + +~~~bash +nitc ./app.nit +~~~ + +Then run it with: + +~~~bash +./app +~~~ + +## System configuration Configure your system for OS X by installing Xcode and brew. Follow the instructions in `README.md` at the root of the repository. Optionally, install ios-sim to run compiled apps in the simulator: `brew install ios-sim` -# Compile and run a simple application +## Compile and run a simple application Let's use the `hello_ios` example in the folder `lib/ios/examples/`. @@ -15,12 +82,91 @@ Compile with: `nitc hello_world.nit` Run in the simulator with: `ios-sim hello_world.app` -# Sample portable applications +* `AppDelegate` - Application interface to the iOS system +* `NSSet_UITouch` - Objective-C `NSSet` of `UITouch` +* `NitGLKView` - OpenGL view controller +* `NitViewController` - View controller associated to an app.nit `Window` +* `TableView` - iOS specific layout using a `UITableView`, works only with simple children views +* `UIApplication` - Graphical application to which events are sent +* `UIButton` - Button on the touch screen +* `UIButtonType` - Style of `UIButton` +* `UIColor` - Color data with opacity +* `UIControl` - Base class for control objects +* `UIEvent` - UIKit event +* `UILabel` - Read-only text view +* `UILayoutConstraintAxis` - Defines the orientation of the arranged views in `UIStackView` +* `UIScrollView` - Support for displaying content larger than the window +* `UIStackView` - Lays out a collection of views in either a column or a row +* `UIStackViewAlignment` - Defines the layout of the arranged views perpendicular to the `UIStackView`'s axis +* `UIStackViewDistribution` - Defines the size and position of the arranged views along the `UIStackView`'s axis +* `UISwitch` - On/Off button +* `UITableView` - View to display and edit hierarchical lists of information +* `UITableViewCell` - Cell of a `UITableViewCell` +* `UITableViewDataSource` - Mediator the data model for a `UITableView` +* `UITableViewDelegate` - Delegate for a `UITableView` to configure selection, sections, cells and more +* `UITableViewStyle` - Style of a `UITableView` +* `UITextField` - Editable text view +* `UITouch` - UIKit touch event +* `UIView` - Rectangular area on the screen +* `UIViewController` - Manages a set of views +* `UIWindow` - Coordinates an app displays on a device screen + +## Sample portable applications See the calculator example at `examples/calculator` and the Tnitter client at `contrib/tnitter/` -for portable applications working on GNU/Linux, OS X, iOS and Android. +for portable applications working on GNU/Linux, OS X, [iOS](ios) and Android. + +Example from `ios::hello_ios`: + +~~~ +# Simple iOS app with a single label +module hello_ios is + example + app_name "Hello iOS" + app_namespace "nit.app.hello_ios" + app_version(0, 5, git_revision) +end + +import ios + +redef class App + redef fun did_finish_launching_with_options + do + return app_delegate.hello_world + end +end + +redef class AppDelegate -# Application icon + # Print and show "Hello World!" + private fun hello_world: Bool in "ObjC" `{ + + // Print to the console + NSLog(@"Hello World!"); + + // Display "Hello world!" on the screen + CGRect frame = [[UIScreen mainScreen] bounds]; + self.window = [[UIWindow alloc] initWithFrame: frame]; + self.window.backgroundColor = [UIColor whiteColor]; + + UILabel *label = [[UILabel alloc] init]; + label.text = @"Hello World!"; + label.center = CGPointMake(100, 100); + [label sizeToFit]; + + // As with `self.window` we must set a `rootViewController` + self.window.rootViewController = [[UIViewController alloc]initWithNibName:nil bundle:nil]; + self.window.rootViewController.view = label; + + [self.window addSubview: label]; + [self.window makeKeyAndVisible]; + + return YES; + `} +end +~~~ + +## Application icon To associate icons to your application, create the directory `ios/AppIcon.appiconset` and fill it with standard icons and `Contents.json`. These files can be generated in a number of different ways: @@ -30,4 +176,12 @@ These files can be generated in a number of different ways: * Using Xcode to assign images to each slot, create the folder and the file `Contents.json`. * Write or modify the file `Contents.json` manually. - It is in Json format and easily readable. + It is in Json format and easily readable. + +## `uikit` + +> Partial wrapper of basic UIKit services. + +## Authors + +This project is maintained by **Alexis Laferrière **. diff --git a/lib/json/README.docdown.md b/lib/json/README.docdown.md new file mode 100644 index 0000000000..60a6f952ca --- /dev/null +++ b/lib/json/README.docdown.md @@ -0,0 +1,273 @@ +# `json` - read and write JSON formatted text + +[[toc: json]] + +These services can be useful to communicate with a remote server or client, +save data locally or even debug and understand the structure of a Nit object. +There is a single API to write [[json | text: JSON]], and three API to read depending on the use case. + +## Write JSON + +[[uml: json | format: svg, mentities: json::json;json::error;json::store;json::dynamic;json::serialization_read;json::serialization_write;json::static]] + +Writing Nit objects to JSON format can be useful to communicate with a remote service, +save data locally or even debug and understand the structure of an object. +There is two related services to write JSON object, the method +`serialize_to_json` and the object `JsonSerializer`. +The method `serialize_to_json` is actually a shortcut to `JsonSerializer`, both +share the same features. + +### Write plain JSON + +Passing the argument `plain=true` to `serialize_to_json` generates plain and clean JSON. +This format is non-Nit program, it cannot be fully deserialized back to Nit objects. +The argument `pretty=true` generates JSON for humans, with more spaces and line breaks. + +The Nit objects to write must subclass `Serializable` and implement its services. +Most classes from the `core` library are already supported, including collections, numeric values, etc. +For your local objects, you can annotate them with `serialize` to automate subclassing +`Serializable` and the implementation of its services. + +#### Example + +~~~ +import json + +class Person + serialize + + var name: String + var year_of_birth: Int + var next_of_kin: nullable Person +end + +var bob = new Person("Bob", 1986) +assert bob.serialize_to_json(pretty=true, plain=true) == """ +{ + "name": "Bob", + "year_of_birth": 1986, + "next_of_kin": null +}""" + +var alice = new Person("Alice", 1978, bob) +assert alice.serialize_to_json(pretty=true, plain=true) == """ +{ + "name": "Alice", + "year_of_birth": 1978, + "next_of_kin": { + "name": "Bob", + "year_of_birth": 1986, + "next_of_kin": null + } +}""" + +# You can also build JSON objects as a `Map` +var charlie = new Map[String, nullable Serializable] +charlie["name"] = "Charlie" +charlie["year_of_birth"] = 1968 +charlie["next_of_kin"] = alice +assert charlie.serialize_to_json(pretty=true, plain=true) == """ +{ + "name": "Charlie", + "year_of_birth": 1968, + "next_of_kin": { + "name": "Alice", + "year_of_birth": 1978, + "next_of_kin": { + "name": "Bob", + "year_of_birth": 1986, + "next_of_kin": null + } + } +}""" +~~~ + +### Write JSON with metadata + +By default, `serialize_to_json` and `JsonSerializer` include metadate in the generated JSON. +This metadata is used by `JsonDeserializer` when reading the JSON code to recreate +the Nit object with the exact original type. +The metadata allows to avoid repeating an object and its resolves cycles in the serialized objects. + +For more information on Nit serialization, see: ../serialization/README.md + +## Read JSON + +There are a total of 3 API to read JSON: + +* `JsonDeserializer` reads JSON to recreate complex Nit objects (discussed here), +* the module [[json::dynamic | text: `json::dynamic`]] provides an easy API to explore [[json::json | text: JSON]] objects, +* the module [[json::static | text: `json::static`]] offers a low-level service to parse JSON and create basic Nit objects. + +The class [[json::JsonDeserializer | text: `JsonDeserializer`]] reads JSON code to recreate objects. +It can use the metadata in the JSON code, to recreate precise Nit objects. +Otherwise, JSON objects are recreated to simple Nit types: `Map`, `Array`, etc. +Errors are reported to the attribute `JsonDeserializer::errors`. + +The type to recreate is either declared or inferred: + +1. The JSON object defines a `__class` key with the name of the Nit class as value. + This attribute is generated by the [[json::JsonSerializer | text: `JsonSerializer`]] with other metadata, + it can also be specified by other external tools. +2. A refinement of `JsonDeserializer::class_name_heuristic` identifies the Nit class. +3. If all else fails, `JsonDeserializer` uses the static type of the attribute, + or the type name passed to `deserialize`. + +The method `deserialize_json` is a shortcut to `JsonDeserializer` which prints +errors to the console. It is fit only for small scripts and other quick and dirty usage. + +[[features: json | mentities: json::json;json::error;json::store;json::dynamic;json::serialization_read;json::serialization_write;json::static]] + +[[features: json | mentities: json::JsonParseError;json::JsonSequenceRead;json::JsonKeyError;json::JsonMapRead;json::JsonArray;json::JsonObject;json::JsonStore;json::JsonDeserializer;json::JsonSerializer;json::JSONStringParser;json::JsonValue]] + +### [[sign: json::dynamic]] + +> [[doc: json::dynamic]] + +#### [[sign: json::JsonValue]] + +> [[doc: json::JsonValue]] + +### [[sign: json::static]] + +> [[doc: json::static]] + +## [[sign: json::store]] + +> [[doc: json::store]] + +#### Example + +~~~ +import json + +class Triangle + serialize + + var corners = new Array[Point] + redef var to_s is serialize_as("name") +end + +class Point + serialize + + var x: Int + var y: Int +end + +# Metadata on each JSON object tells the deserializer what is its Nit type, +# and it supports special types such as generic collections. +var json_with_metadata = """{ + "__class": "Triangle", + "corners": {"__class": "Array[Point]", + "__items": [{"__class": "Point", "x": 0, "y": 0}, + {"__class": "Point", "x": 3, "y": 0}, + {"__class": "Point", "x": 2, "y": 2}]}, + "name": "some triangle" +}""" + +var deserializer = new JsonDeserializer(json_with_metadata) +var object = deserializer.deserialize +assert deserializer.errors.is_empty +assert object != null + +# However most non-Nit services won't add the metadata and instead produce plain JSON. +# Without a "__class", the deserializer relies on `class_name_heuristic` and the static type. +# The type of the root object to deserialize can be specified by an argument passed to `deserialize`. +var plain_json = """{ + "corners": [{"x": 0, "y": 0}, + {"x": 3, "y": 0}, + {"x": 2, "y": 2}], + "name": "the same triangle" +}""" + +deserializer = new JsonDeserializer(plain_json) +object = deserializer.deserialize("Triangle") +assert deserializer.errors.is_empty # If false, `object` is invalid +~~~ + +#### Missing attributes and default values + +When reading JSON, some attributes expected by Nit classes may be missing. +The JSON object may come from an external API using optional attributes or +from a previous version of your program without the attributes. +When an attribute is not found, the deserialization engine acts in one of three ways: + +1. If the attribute has a default value or if it is annotated by `lazy`, + the engine leave the attribute to the default value. No error is raised. +2. If the static type of the attribute is nullable, the engine sets + the attribute to `null`. No error is raised. +3. Otherwise, the engine raises an error and does not set the attribute. + The caller must check for `errors` and must not read from the attribute. + +~~~ +import json + +class MyConfig + serialize + + var width: Int # Must be in JSON or an error is raised + var height = 4 + var volume_level = 8 is lazy + var player_name: nullable String + var tmp_dir: nullable String = "/tmp" is lazy +end + +# --- +# JSON object with all expected attributes -> OK +var plain_json = """ +{ + "width": 11, + "height": 22, + "volume_level": 33, + "player_name": "Alice", + "tmp_dir": null +}""" +var deserializer = new JsonDeserializer(plain_json) +var obj = deserializer.deserialize("MyConfig") + +assert deserializer.errors.is_empty +assert obj isa MyConfig +assert obj.width == 11 +assert obj.height == 22 +assert obj.volume_level == 33 +assert obj.player_name == "Alice" +assert obj.tmp_dir == null + +# --- +# JSON object missing optional attributes -> OK +plain_json = """ +{ + "width": 11 +}""" +deserializer = new JsonDeserializer(plain_json) +obj = deserializer.deserialize("MyConfig") + +assert deserializer.errors.is_empty +assert obj isa MyConfig +assert obj.width == 11 +assert obj.height == 4 +assert obj.volume_level == 8 +assert obj.player_name == null +assert obj.tmp_dir == "/tmp" + +# --- +# JSON object missing the mandatory attribute -> Error +plain_json = """ +{ + "player_name": "Bob", +}""" +deserializer = new JsonDeserializer(plain_json) +obj = deserializer.deserialize("MyConfig") + +# There's an error, `obj` is partial +assert deserializer.errors.length == 1 + +# Still, we can access valid attributes +assert obj isa MyConfig +assert obj.player_name == "Bob" +~~~ + +## Authors + +This project is maintained by [[ini-maintainer: json]]. diff --git a/lib/json/README.md b/lib/json/README.md index 94539c3e0e..44f807a03d 100644 --- a/lib/json/README.md +++ b/lib/json/README.md @@ -1,10 +1,25 @@ -read and write JSON formatted text +# `json` - read and write JSON formatted text + +* [Write JSON](#Write-JSON) +* [Write plain JSON](#Write-plain-JSON) +* [Example](#Example) +* [Write JSON with metadata](#Write-JSON-with-metadata) +* [Read JSON](#Read-JSON) +* [`dynamic`](#`dynamic`) +* [`JsonValue`](#`JsonValue`) +* [`static`](#`static`) +* [`store`](#`store`) +* [Example](#Example) +* [Missing attributes and default values](#Missing-attributes-and-default-values) +* [Authors](#Authors) These services can be useful to communicate with a remote server or client, save data locally or even debug and understand the structure of a Nit object. -There is a single API to write JSON, and three API to read depending on the use case. +There is a single API to write [JSON](json), and three API to read depending on the use case. -# Write JSON +## Write JSON + +![Diagram for `json`](uml-json.svg) Writing Nit objects to JSON format can be useful to communicate with a remote service, save data locally or even debug and understand the structure of an object. @@ -13,7 +28,7 @@ There is two related services to write JSON object, the method The method `serialize_to_json` is actually a shortcut to `JsonSerializer`, both share the same features. -## Write plain JSON +### Write plain JSON Passing the argument `plain=true` to `serialize_to_json` generates plain and clean JSON. This format is non-Nit program, it cannot be fully deserialized back to Nit objects. @@ -24,9 +39,11 @@ Most classes from the `core` library are already supported, including collection For your local objects, you can annotate them with `serialize` to automate subclassing `Serializable` and the implementation of its services. -### Example +#### Example ~~~ +import json + class Person serialize @@ -76,7 +93,7 @@ assert charlie.serialize_to_json(pretty=true, plain=true) == """ }""" ~~~ -## Write JSON with metadata +### Write JSON with metadata By default, `serialize_to_json` and `JsonSerializer` include metadate in the generated JSON. This metadata is used by `JsonDeserializer` when reading the JSON code to recreate @@ -85,15 +102,15 @@ The metadata allows to avoid repeating an object and its resolves cycles in the For more information on Nit serialization, see: ../serialization/README.md - -# Read JSON +## Read JSON There are a total of 3 API to read JSON: + * `JsonDeserializer` reads JSON to recreate complex Nit objects (discussed here), -* the module `json::dynamic` provides an easy API to explore JSON objects, -* the module `json::static` offers a low-level service to parse JSON and create basic Nit objects. +* the module [`json::dynamic`](json::dynamic) provides an easy API to explore [JSON](json::json) objects, +* the module [`json::static`](json::static) offers a low-level service to parse JSON and create basic Nit objects. -The class `JsonDeserializer` reads JSON code to recreate objects. +The class [`JsonDeserializer`](json::JsonDeserializer) reads JSON code to recreate objects. It can use the metadata in the JSON code, to recreate precise Nit objects. Otherwise, JSON objects are recreated to simple Nit types: `Map`, `Array`, etc. Errors are reported to the attribute `JsonDeserializer::errors`. @@ -101,7 +118,7 @@ Errors are reported to the attribute `JsonDeserializer::errors`. The type to recreate is either declared or inferred: 1. The JSON object defines a `__class` key with the name of the Nit class as value. - This attribute is generated by the `JsonSerializer` with other metadata, + This attribute is generated by the [`JsonSerializer`](json::JsonSerializer) with other metadata, it can also be specified by other external tools. 2. A refinement of `JsonDeserializer::class_name_heuristic` identifies the Nit class. 3. If all else fails, `JsonDeserializer` uses the static type of the attribute, @@ -110,9 +127,180 @@ The type to recreate is either declared or inferred: The method `deserialize_json` is a shortcut to `JsonDeserializer` which prints errors to the console. It is fit only for small scripts and other quick and dirty usage. -### Example +* `dynamic` - Dynamic interface to read values from JSON strings + +* `error` - Intro `JsonParseError` which is exposed by all JSON reading APIs + +* `json` - Read and write JSON formatted text using the standard serialization services + +* `serialization_read` - Services to read JSON: `deserialize_json` and `JsonDeserializer` + +* `serialization_write` - Services to write Nit objects to JSON strings: `serialize_to_json` and `JsonSerializer` + +* `static` - Static interface to read Nit objects from JSON strings + +* `store` - Store and load json data. + +* `JSONStringParser` - A simple ad-hoc JSON parser + +* `JsonArray` - A JSON array. + +* `JsonDeserializer` - Deserializer from a Json string. + +* `JsonKeyError` - Key access error + +* `JsonMapRead` - A map that can be translated into a JSON object. + +* `JsonObject` - A JSON Object. + +* `JsonParseError` - JSON format error at parsing + +* `JsonSequenceRead` - A sequence that can be translated into a JSON array. + +* `JsonSerializer` - Serializer of Nit objects to Json string. + +* `JsonStore` - A JsonStore can save and load json data from file system. + +* `JsonValue` - Dynamic wrapper of a JSON value, created by `Text::to_json_value` + +### `dynamic` + +> Most services are in `JsonValue`, which is created by `Text::to_json_value`. + +#### `JsonValue` + +> Provides high-level services to explore JSON objects with minimal overhead +> dealing with static types. Use this class to manipulate the JSON data first +> and check for errors only before using the resulting data. + +For example, given: + +~~~ +var json_src = """ +{ + "i": 123, + "m": { + "t": true, + "f": false + }, + "a": ["zero", "one", "two"] +}""" +var json_value = json_src.to_json_value # Parse src to a `JsonValue` +~~~ + +Access array or map values using their indices. + +~~~ +var target_int = json_value["i"] +assert target_int.is_int # Check for error and expected type +assert target_int.to_i == 123 # Use the value +~~~ + +Use `get` to reach a value nested in multiple objects. + +~~~ +var target_str = json_value.get("a.0") +assert target_str.is_string # Check for error and expected type +assert target_str.to_s == "zero" # Use the value +~~~ + +This API is useful for scripts and when you need only a few values from a +JSON object. To access many values or check the conformity of the JSON +beforehand, use the `json` serialization services. + +### `static` + +> `Text::parse_json` returns a simple Nit object from the JSON source. +> This object can then be type checked as usual with `isa` and `as`. + +## `store` + +> This simple system can be used to store and retrieve json data. + +## Usage + +### Initialization + +JsonStore use the file system to store and load json files. + +For initialization you need to give the directory used in the +file system to save objects. + +~~~ +var store = new JsonStore("store_dir") +~~~ + +### Documents + +With JsonStore you manage *documents*. +Documents are simple json files that can be stored and loaded from json store. + +JsonStore can store documents of type JsonObject and JsonArray. + +~~~ +var red = new JsonObject +red["name"] = "red" +red["code"] = "FF0000" +~~~ + +Data are stored under a *key*. +This is the path to the document from `JsonStore::store_dir` +without the `.json` extension. + +Examples: + +* key `document` will store data under `store_dir / "document.json"` +* key `collection/data` will store data under `store_dir / "collection/data.json"` + +~~~ +var key = "colors/red" +~~~ + +Store the object. + +~~~ +store.store_object(key, red) +~~~ + +Load the object. + +~~~ +assert store.has_key(key) +var obj = store.load_object(key) +assert obj["name"] == obj["name"] +~~~ + +### Collections + +A collection is a set of documents stored under the same path. + +~~~ +var green = new JsonObject +green["name"] = "green" +green["code"] = "00FF00" +store.store_object("colors/green", green) + +assert store.has_collection("colors") + +var col = store.list_collection("colors") +assert col.length == 2 +assert col.has("green") +assert col.has("red") +~~~ + +### Clearing store + +You can delete all the data contained in the `JsonStore::store_dir` with `clear`. ~~~ +store.clear +~~~ + +#### Example + +~~~ +import json + class Triangle serialize @@ -158,7 +346,7 @@ object = deserializer.deserialize("Triangle") assert deserializer.errors.is_empty # If false, `object` is invalid ~~~ -### Missing attributes and default values +#### Missing attributes and default values When reading JSON, some attributes expected by Nit classes may be missing. The JSON object may come from an external API using optional attributes or @@ -173,6 +361,8 @@ When an attribute is not found, the deserialization engine acts in one of three The caller must check for `errors` and must not read from the attribute. ~~~ +import json + class MyConfig serialize @@ -237,3 +427,7 @@ assert deserializer.errors.length == 1 assert obj isa MyConfig assert obj.player_name == "Bob" ~~~ + +## Authors + +This project is maintained by **Alexis Laferrière **. diff --git a/lib/markdown/README.docdown.md b/lib/markdown/README.docdown.md new file mode 100644 index 0000000000..5079be14c9 --- /dev/null +++ b/lib/markdown/README.docdown.md @@ -0,0 +1,65 @@ +# `markdown` - A markdown parser for Nit + +[[toc: markdown]] + +[[markdown | text: Markdown]] documentation can be found in http://daringfireball.net/projects/markdown/. +This parser is inspired by the famous TxtMark for Java (https://github.com/rjeschke/txtmark). + +## Getting Started + +These instructions will get you a copy of the project up and running on your local machine. + +### Dependencies + +This project requires the following packages: + +[[parents: markdown]] + +### Run `nitmd` + +[[uml: markdown | format: svg, mentities: markdown::nitmd;markdown::wikilinks;markdown::man;markdown::decorators;markdown::markdown]] + +`nitmd` can be used as a standalone tool: + +[[main-compile: markdown::nitmd]] + +Then run it with: + +[[main-run: markdown::nitmd]] + +Options: + +[[main-opts: markdown::nitmd]] + +Or you can use it programmatically by importing the [[markdown::markdown | text: `markdown`]] module. + +[[features: markdown | mentities: markdown::nitmd;markdown::wikilinks;markdown::man;markdown::decorators;markdown::markdown]] + +## Differences with Markdown specification + +This parser passes all tests inside http://daringfireball.net/projects/downloads/MarkdownTest_1.0_2007-05-09.tgz execpt of two: + +1. Images.text: fails because this parser doesn't produce empty 'title' image attributes. +2. Literal quotes in titles.text: because markdown accepts unescaped quotes in titles and this is wrong. + +## [[sign: markdown::MarkdownProcessor]] + +> [[doc: markdown::MarkdownProcessor]] + +## [[sign: markdown::wikilinks]] + +> [[doc: markdown::wikilinks]] + +## Running the tests + +The NitUnit test suite can be found in `tests`. + +Minimalists tests are prefixed with `test_process_*`. All tests from daringfireball are prefixed with `test_daring*`. + +Run the test suite: + +[[testing: markdown]] + +## Authors + +This project is maintained by [[ini-maintainer: markdown]]. diff --git a/lib/markdown/README.md b/lib/markdown/README.md index 7068bb0b49..f247532847 100644 --- a/lib/markdown/README.md +++ b/lib/markdown/README.md @@ -1,17 +1,58 @@ -A markdown parser for Nit. +# `markdown` - A markdown parser for Nit -Markdown documentation can be found in http://daringfireball.net/projects/markdown/. +* [Getting Started](#Getting-Started) +* [Dependencies](#Dependencies) +* [Run `nitmd`](#Run-`nitmd`) +* [Differences with Markdown specification](#Differences-with-Markdown-specification) +* [`MarkdownProcessor`](#`MarkdownProcessor`) +* [`wikilinks`](#`wikilinks`) +* [Running the tests](#Running-the-tests) +* [Authors](#Authors) + +[Markdown](markdown) documentation can be found in http://daringfireball.net/projects/markdown/. This parser is inspired by the famous TxtMark for Java (https://github.com/rjeschke/txtmark). -## Usage +## Getting Started + +These instructions will get you a copy of the project up and running on your local machine. + +### Dependencies + +This project requires the following packages: + +* `config` - Configuration options for nit tools and apps +* `template` - Basic template system + +### Run `nitmd` + +![Diagram for `markdown`](uml-markdown-2.svg) `nitmd` can be used as a standalone tool: ~~~bash -$ nitmd file.md +nitc ./nitmd.nit +~~~ + +Then run it with: + +~~~bash +nitmd [-t format] +~~~ + +Options: + +~~~bash +* -h, -?, --help Show this help message. +* -t, --to Specify output format (html, md, man). ~~~ -Or you can use it programmatically by importing the `markdown` module. +Or you can use it programmatically by importing the [`markdown`](markdown::markdown) module. + +* `decorators` - Decorators for `markdown` parsing. +* `man` - Simple *groff* decorator restricted for manpages. +* `markdown` - Markdown parsing. +* `nitmd` - A Markdown parser for Nit. +* `wikilinks` - Wikilinks handling. ## Differences with Markdown specification @@ -20,14 +61,39 @@ This parser passes all tests inside http://daringfireball.net/projects/downloads 1. Images.text: fails because this parser doesn't produce empty 'title' image attributes. 2. Literal quotes in titles.text: because markdown accepts unescaped quotes in titles and this is wrong. -## Testing +## `MarkdownProcessor` + +> Blocks are then outputed by an `MarkdownEmitter`. + +Usage: + +var proc = new MarkdownProcessor +var html = proc.process("**Hello World!**") +assert html == "

Hello World!

\n" + +SEE: `String::md_to_html` for a shortcut. -The NitUnit test suite can be found in `test_markdown.nit`. +## `wikilinks` -Minimalists tests are prefixed with `process_*`. All tests from daringfireball are prefixed with `process_daring*`. +> Wikilinks are on the form `[[link]]`. +> They can also contain a custom title with the syntax `[[title|link]]`. + +By importing this module, you enable the `MarkdownProcessor` to recognize +`TokenWikiLink` but nothing will happen until you define a +`Decorator::add_wikilink` customized to your applciation domain. + +## Running the tests + +The NitUnit test suite can be found in `tests`. + +Minimalists tests are prefixed with `test_process_*`. All tests from daringfireball are prefixed with `test_daring*`. Run the test suite: ~~~bash -$ nitunit lib/markdown/markdown.nit -t lib/markdown/test_markdown.nit +nitunit . ~~~ + +## Authors + +This project is maintained by **Alexandre Terrasa **. diff --git a/lib/nitcorn/README.docdown.md b/lib/nitcorn/README.docdown.md new file mode 100644 index 0000000000..bde764782a --- /dev/null +++ b/lib/nitcorn/README.docdown.md @@ -0,0 +1,134 @@ +# `nitcorn` - Lightweight framework for Web applications development + +[[toc: nitcorn]] + +> [[doc: nitcorn::nitcorn]] + +## Features + +[[uml: nitcorn | format: svg, mentities: nitcorn::pthreads;nitcorn::nitcorn;nitcorn::signal_handler;nitcorn::token;nitcorn::proxy;nitcorn::server_config;nitcorn::restful;nitcorn::sessions;nitcorn::http_errors;nitcorn::log;nitcorn::http_request_buffer;nitcorn::reactor;nitcorn::vararg_routes;nitcorn::file_server;nitcorn::http_request;nitcorn::http_response;nitcorn::media_types]] + +Dynamic content is served by subclassing [[nitcorn::Action | text: `Action`]] and implementing `answer`. +This method receives an [[nitcorn::HttpRequest | text: `HttpRequest`]] and must return an [[nitcorn::HttpResponse | text: `HttpResponse`]]. +_nitcorn_ provides [[nitcorn::FileServer | text: `FileServer`]], a simple `Action` to serve static files. + +`HttpRequest` contains the GET and POST arguments as well as [[nitcorn::Session | text: session]] data it one exists. +The produced `HttpResponse` should contain the HTTP status code, the body, +session data to preserve or create a session, and optionally list files to append. + +Each `Action` may be associated to many instances of [[nitcorn::Route | text: `Route`]]. +These [[nitcorn::Routes | text: routes]] can simply identify the root of a service, +but also define parameters within the URI. + +_nitcorn_ instances are configured dynamically in Nit code with the [[nitcorn::Interface | text: interfaces]] and routes created as needed. + +_nitcorn_ plays well with other Nit services and tools such as `serialization`, `mongodb`, `sqlite` and `nitiwiki`. +It also benefits from the full power of the Nit language: +class refinement can be used to customize default services and merge many applications in a single server, +and the FFI enables calling services in different languages. + +Example from `nitcorn::nitcorn_reverse_proxy`: + +[[code: nitcorn::nitcorn_reverse_proxy]] + +[[features: nitcorn | mentities: nitcorn::pthreads;nitcorn::nitcorn;nitcorn::signal_handler;nitcorn::token;nitcorn::proxy;nitcorn::server_config;nitcorn::restful;nitcorn::sessions;nitcorn::http_errors;nitcorn::log;nitcorn::http_request_buffer;nitcorn::reactor;nitcorn::vararg_routes;nitcorn::file_server;nitcorn::http_request;nitcorn::http_response;nitcorn::media_types]] + +## Examples + +A minimal example follows with a custom `Action` and using `FileServer`. + +More general examples are available at `lib/nitcorn/examples/`. +For an example of a larger project merging many _[[nitcorn]]_ applications into one server, +take a look at the configuration of `http://xymus.net/` at `../contrib/xymus_net/xymus_net.nit`. + +Larger projects using _nitcorn_ can be found in the `contrib/` folder: + +* _opportunity_ is a meetup planner heavily based on _nitcorn_. +* _tnitter_ is a micro-blogging platform with a simple Web and RESTful interface. +* _benitlux_ uses a custom `Action` to subscribe people to a mailing list and define a RESTful [[nitcorn::Interfaces | text: interface]]. + Example from `nitcorn::nitcorn_hello_world`: + +[[code: nitcorn::nitcorn_hello_world]] + +### Simple hello world server + +~~~ +import nitcorn + +# Simple Action to display the Hello World page and the get arguments +class HelloWorldAction + super Action + + redef fun answer(http_request, turi) + do + var title = "Hello World!" + var args = http_request.get_args.join(",", ":") + + var response = new HttpResponse(200) + response.body = """ + + + + + {{{title}}} + + +

{{{title}}}

+

GET args: {{{args}}}

+ +""" + return response + end +end + +# Listen on `localhost:8080` +var vh = new VirtualHost("localhost:8080") + +# Serve `http://localhost:8080/hello.html` with our custom action +vh.routes.add new Route("/hello.html", new HelloWorldAction) + +# Serve everything else under `http://localhost:8080/` using a `FileServer` with a root at "/var/www/" +vh.routes.add new Route(null, new FileServer("/var/www/")) + +# Launch server +var factory = new HttpFactory.and_libevent +factory.config.virtual_hosts.add vh +factory.run +~~~ + +## [[sign: nitcorn::server_config]] + +> [[doc: nitcorn::server_config]] + +## [[sign: nitcorn::restful]] + +> [[doc: nitcorn::restful]] + +## [[sign: nitcorn::sessions]] + +> [[doc: nitcorn::sessions]] + +## [[sign: nitcorn::log]] + +> [[doc: nitcorn::log]] + +## [[sign: nitcorn::vararg_routes]] + +> [[doc: nitcorn::vararg_routes]] + +## Credits + +This nitcorn library is a fork from an independent project originally created in 2013 by +Jean-Philippe Caissy, Guillaume Auger, Frederic Sevillano, Justin Michaud-Ouellette, +Stephan Michaud and Maxime Bélanger. + +It has been adapted to a library, and is currently maintained, by Alexis Laferrière. + +This project is maintained by [[ini-maintainer: nitcorn]]. + +Thanks to the contribution of: +[[ini-contributors: nitcorn]] + +Example from `nitcorn::htcpcp_server`: + +[[code: nitcorn::htcpcp_server]] diff --git a/lib/nitcorn/README.md b/lib/nitcorn/README.md index 2db66b894a..ee56d5ae07 100644 --- a/lib/nitcorn/README.md +++ b/lib/nitcorn/README.md @@ -1,40 +1,208 @@ -Lightweight framework for Web applications development +# `nitcorn` - Lightweight framework for Web applications development -# Features +* [Features](#Features) +* [Examples](#Examples) +* [Simple hello world server](#Simple-hello-world-server) +* [`server_config`](#`server_config`) +* [`restful`](#`restful`) +* [`sessions`](#`sessions`) +* [`log`](#`log`) +* [`vararg_routes`](#`vararg_routes`) +* [Credits](#Credits) -Dynamic content is served by subclassing `Action` and implementing `answer`. -This method receives an `HttpRequest` and must return an `HttpResponse`. -_nitcorn_ provides `FileServer`, a simple `Action` to serve static files. +> The main classes are: -`HttpRequest` contains the GET and POST arguments as well as session data it one exists. +* `Action` to answer to requests. +* `Route` to represent a path to an action. +* `VirtualHost` to listen on a specific interface and behave accordingly +* `HttpFactory` which is the base dispatcher class. + +Basic usage example: + +~~~~ +class MyAction + super Action + + redef fun answer(http_request, turi) + do + var response = new HttpResponse(200) + response.body = """ + + + + Hello World + + +

Hello World

+ +""" + return response + end +end + +# Listen to port 8080 on all interfaces +var vh = new VirtualHost("0.0.0.0:8080") + +# Serve index.html with our custom handler +vh.routes.add new Route("/index.html", new MyAction) + +# Serve everything else with a standard `FileServer` +vh.routes.add new Route(null, new FileServer("/var/www/")) + +var factory = new HttpFactory.and_libevent +factory.config.virtual_hosts.add vh +factory.run +~~~~ + +## Features + +![Diagram for `nitcorn`](uml-nitcorn-1.svg) + +Dynamic content is served by subclassing [`Action`](nitcorn::Action) and implementing `answer`. +This method receives an [`HttpRequest`](nitcorn::HttpRequest) and must return an [`HttpResponse`](nitcorn::HttpResponse). +_nitcorn_ provides [`FileServer`](nitcorn::FileServer), a simple `Action` to serve static files. + +`HttpRequest` contains the GET and POST arguments as well as [session](nitcorn::Session) data it one exists. The produced `HttpResponse` should contain the HTTP status code, the body, session data to preserve or create a session, and optionally list files to append. -Each `Action` may be associated to many instances of `Route`. -These routes can simply identify the root of a service, +Each `Action` may be associated to many instances of [`Route`](nitcorn::Route). +These [routes](nitcorn::Routes) can simply identify the root of a service, but also define parameters within the URI. -_nitcorn_ instances are configured dynamically in Nit code with the interfaces and routes created as needed. +_nitcorn_ instances are configured dynamically in Nit code with the [interfaces](nitcorn::Interface) and routes created as needed. _nitcorn_ plays well with other Nit services and tools such as `serialization`, `mongodb`, `sqlite` and `nitiwiki`. It also benefits from the full power of the Nit language: class refinement can be used to customize default services and merge many applications in a single server, and the FFI enables calling services in different languages. -# Examples +Example from `nitcorn::nitcorn_reverse_proxy`: + +~~~ +# Minimal example using a `ProxyAction` +module nitcorn_reverse_proxy is example + +import nitcorn::proxy + +# Create the virtualhost for your nitcorn server +var vh = new VirtualHost("localhost:8080") + +# Create the interface to represent your proxy target +var proxy_interface = new Interface("localhost", 31337) + +# Add your action as usual +vh.routes.add new Route("/", new ProxyAction(proxy_interface)) + +# Let it be (serve) +var factory = new HttpFactory.and_libevent +factory.config.virtual_hosts.add vh +factory.run +~~~ + +* `file_server` - Provides the `FileServer` action, which is a standard and minimal file server +* `http_errors` - Offers `ErrorTemplate` to display error pages +* `http_request` - Provides the `HttpRequest` class and services to create it +* `http_request_buffer` - Http request parsing for buffered inputs. +* `http_response` - Provides the `HttpResponse` class and `http_status_codes` +* `log` - Services inserting a timestamp in all prints and to log each requests +* `media_types` - Services to identify Internet media types (or MIME types, Content-types) +* `nitcorn` - The nitcorn Web server framework creates server-side Web apps in Nit +* `proxy` - Provides the `ProxyAction` action, which redirects requests to another interface +* `pthreads` - Activate the use of pthreads with `nitcorn` +* `reactor` - Core of the `nitcorn` project, provides `HttpFactory` and `Action` +* `restful` - Support module for the `nitrestful` tool and the `restful` annotation +* `server_config` - Classes and services to configure the server +* `sessions` - Automated session management +* `signal_handler` - Handle SIGINT and SIGTERM to close the server after all active events +* `token` - Simple `generate_token` service, independent of the rest of the nitcorn framework +* `vararg_routes` - Routes with parameters. + +## Examples A minimal example follows with a custom `Action` and using `FileServer`. More general examples are available at `lib/nitcorn/examples/`. -For an example of a larger project merging many _nitcorn_ applications into one server, +For an example of a larger project merging many _[nitcorn](nitcorn)_ applications into one server, take a look at the configuration of `http://xymus.net/` at `../contrib/xymus_net/xymus_net.nit`. Larger projects using _nitcorn_ can be found in the `contrib/` folder: + * _opportunity_ is a meetup planner heavily based on _nitcorn_. * _tnitter_ is a micro-blogging platform with a simple Web and RESTful interface. -* _benitlux_ uses a custom `Action` to subscribe people to a mailing list and define a RESTful interface. +* _benitlux_ uses a custom `Action` to subscribe people to a mailing list and define a RESTful [interface](nitcorn::Interfaces). + Example from `nitcorn::nitcorn_hello_world`: + +~~~ +# Hello World Web server example +# +# The main page, `index.html`, is served dynamicly with `MyAction`. +# The rest of the Web site fetches files from the local directory +# `www/hello_world/`. +module nitcorn_hello_world is example + +import nitcorn -## Simple hello world server +# An action that responds by displaying a static html content. +class StaticAction + super Action + + redef fun answer(http_request, turi) + do + var response = new HttpResponse(200) + var title = "Hello World from Nitcorn!" + response.body = """ + + + + + + {{{title}}} + + +
+

{{{title}}}

+

See also a directory.

+
+ +""" + return response + end +end + +# An action that uses parameterized uris to customize the output. +class ParamAction + super Action + + redef fun answer(http_request, turi) + do + var response = new HttpResponse(200) + var name = http_request.param("name") + if name == null then + response.body = "No name..." + else + response.body = "Hello {name}" + end + return response + end +end + + +var vh = new VirtualHost("localhost:8080") + +# Serve index.html with our custom handler +vh.routes.add new Route("/index.html", new StaticAction) +vh.routes.add new Route("/hello/:name", new ParamAction) + +# Serve everything else with a standard `FileServer` with a root at "www/hello_world/" +vh.routes.add new Route(null, new FileServer("www/hello_world/")) + +var factory = new HttpFactory.and_libevent +factory.config.virtual_hosts.add vh +factory.run +~~~ + +### Simple hello world server ~~~ import nitcorn @@ -80,10 +248,272 @@ factory.config.virtual_hosts.add vh factory.run ~~~ -# Credits +## `server_config` + +> The classes of interest are `VirtualHost`, `Interface`, `Route` and `Action` + +## `restful` + +> The `restful` annotation is applied on a method to assign it to an HTTP resource. +> The `restful` method must be a property of a subclass of `RestfulAction` and +> return an `HTTPResponse`. +> Once an instance of the class is assigned to a route, the method +> can be invoked as a resource under that route. +> The returned `HTTPResponse` will be sent back to the client. + +The arguments of the method must be deserializable. +So use simple data types like `String`, `Int`, `Float`, etc. +or any other `Serializable` class. +The method is invoked only if all the arguments are correctly passed +in the JSON format by the HTTP client. +There is one exception, `String` arguments are returned as is, +they don't need the surrounding `""`. +If an argument is missing or there a format error, the `answer` method responds to the request. +Arguments that are `nullable` are optional, +if they are missing `null` is passed to the `restful` method. + +The annotation accepts three kinds of arguments, in any order: + +* String literals rename or add an alias for the HTTP resource. + By default, the name of the HTTP resource is the name of the `restful` method. + The first string literal replaces the default name, + while additional string literals add aliases. + +* Ids such as `GET`, `POST`, `PUT` and `DELETE` restrict which HTTP methods + are accepted. By default, all HTTP methods are accepted. + +* The `async` keyword triggers executing calls to this service asynchronously + by the `thread_pool` attribute of the `RestfulAction`. + By default, each call are executed on the same thread in a FIFO order. + +See the example at `lib/nitcorn/examples/restful_annot.nit` or +a real world use case at `contrib/benitlux/src/server/benitlux_controller.nit`. + +The `restful` annotation is implemented by then `nitrestful` tool. +To compile a module (`my_module.nit`) that uses the `restful` annotation: + +* Run `nitrestful my_module.nit` to produce `my_module_rest.nit` +* Link `my_module_rest.nit` at compilation: `nitc my_module.nit -m my_module_rest.nit`. + +## `sessions` + +> When parsing a request, this module associate a pre-existing session +> to the request if there is one. It will also send the required cookie +> with the response if a session has been associated to the response object. + +## `log` + + +## `vararg_routes` + +> Using `vararg_routes`, a `Route` path can contain variable parts +> that will be matched against a `HttpRequest` URL. + +Variable parameters of a route path can be specified using the `:` prefix: + +~~~nitish +var iface = "http://localhost:3000" +var vh = new VirtualHost(iface) +vh.routes.add new Route("/blog/articles/:articleId", new BlogArticleAction) +~~~ + +Route arguments can be accessed from the `HttpRequest` within a nitcorn `Action`: + +~~~nitish +class BlogArticleAction + super Action + + redef fun answer(request, url) do + var param = request.param("articleId") + if param == null then + return new HttpResponse(400) + end + + print url # let's say "/blog/articles/12" + print param # 12 + + return new HttpResponse(200) + end +end +~~~ + +## Route matching + +Route can match variables expression. + +~~~ +# We need an Action to try routes. +class DummyAction super Action end +var action = new DummyAction + +var route = new Route("/users/:id", action) +assert not route.match("/users") +assert route.match("/users/1234") +assert route.match("/users/") # empty id +~~~ + +Route without uri parameters still behave like before. + +~~~ +route = new Route("/users", action) +assert route.match("/users") +assert route.match("/users/1234") +assert not route.match("/issues/1234") +~~~ + +## Route priority + +Priority depends on the order the routes were added to the `Routes` dispatcher. + +~~~ +var host = new VirtualHost("") +var routes = new Routes(host) + +routes.add new Route("/:a/:b/:c", action) +routes.add new Route("/users/:id", action) +routes.add new Route("/:foo", action) + +assert routes["/a/b/c"].path == "/:a/:b/:c" +assert routes["/a/b/c/d"].path == "/:a/:b/:c" +assert routes["/users/1234/foo"].path == "/:a/:b/:c" + +assert routes["/users/"].path == "/users/:id" +assert routes["users/"].path == "/users/:id" +assert routes["/users/1234"].path == "/users/:id" + +assert routes["/users"].path == "/:foo" +assert routes["/"].path == "/:foo" +assert routes[""].path == "/:foo" +~~~ + +## Accessing uri parameter and values + +Parameters can be accessed by parsing the uri. + +~~~ +route = new Route("/users/:id", action) +var params = route.parse_params("/users/1234") +assert params.has_key("id") +assert not params.has_key("foo") +assert params["id"] == "1234" +~~~ + +Or from the `HttpRequest`. + +~~~ +route = new Route("/users/:id", action) +var req = new HttpRequest +req.uri_params = route.parse_params("/users/1234") +assert req.params == ["id"] +assert req.param("id") == "1234" +assert req.param("foo") == null +~~~ + +## Credits This nitcorn library is a fork from an independent project originally created in 2013 by Jean-Philippe Caissy, Guillaume Auger, Frederic Sevillano, Justin Michaud-Ouellette, Stephan Michaud and Maxime Bélanger. It has been adapted to a library, and is currently maintained, by Alexis Laferrière. + +This project is maintained by **Alexis Laferrière **. + +Thanks to the contribution of: + +* **Jean-Philippe Caissy ** +* **Guillaume Auger** +* **Frederic Sevillano** +* **Justin Michaud-Ouellette** +* **Stephan Michaud** +* **Maxime Bélanger** + +Example from `nitcorn::htcpcp_server`: + +~~~ +# A server that implements HTCPCP. At the moment there are no additions. +module htcpcp_server is example + +import nitcorn + +# Nitcorn Action used to answer requests. +class HTCPCPAction + super Action + + # Brewing status. + var brewing = false + + # Teapot status. + var is_teapot = false + + redef fun answer(http_request, turi) do + var message: String + var method = http_request.method + var response: HttpResponse + + if is_teapot then + response = new HttpResponse(418) + response.body = "I'm a teapot!\n" + response.header["Content-Type"] = "text" + return response + end + + if method == "POST" or method == "BREW" then + if brewing then + message = "Pot Busy" + response = new HttpResponse(400) + else + message = "Brewing a new pot of coffee\n" + brewing = true + response = new HttpResponse(200) + end + else if method == "WHEN" then + if brewing then + message = "Stopped adding milk, your coffee is ready!\n" + brewing = false + response = new HttpResponse(200) + else + message = "There is no coffee brewing!\n" + response = new HttpResponse(405) + end + else if method == "PROPFIND" or method == "GET" then + if brewing then + message = "The pot is busy\n" + else + message = "The pot is ready to brew more coffee\n" + end + response = new HttpResponse(200) + else + message = "Unknown method: {method}" + brewing = false + response = new HttpResponse(405) + end + + response.header["Content-Type"] = "text" + response.body = message + + return response + end +end + +# Nitcorn server. +class HTCPCServer + + # Port to listen to. + var port: Int + + # Start listening. + fun run do + var vh = new VirtualHost("localhost:{port}") + vh.routes.add new Route("/", new HTCPCPAction) + var factory = new HttpFactory.and_libevent + factory.config.virtual_hosts.add vh + print "Nit4Coffee is now running at port: {port}" + factory.run + end +end + +var server = new HTCPCServer(1227) + +server.run +~~~ diff --git a/lib/nlp/README.docdown.md b/lib/nlp/README.docdown.md new file mode 100644 index 0000000000..64c1122357 --- /dev/null +++ b/lib/nlp/README.docdown.md @@ -0,0 +1,101 @@ +# `nlp` - Nit wrapper for Stanford CoreNLP + +[[toc: nlp]] + +Stanford CoreNLP provides a set of natural language analysis tools which can take +raw text input and give the base forms of words, their parts of speech, whether +they are names of companies, people, etc., normalize dates, times, and numeric +quantities, and mark up the structure of sentences in terms of phrases and word +dependencies, indicate which noun phrases refer to the same entities, indicate +sentiment, etc. + +This wrapper needs the Stanford CoreNLP jars that run on Java 1.8+. + +> [[doc: nlp::nlp]] + +> [[features: nlp | mentities: nlp::NLPSentence;nlp::NLPFileIndex;nlp::NLPDocument;nlp::NLPJavaProcessor;nlp::NLPClient;nlp::NLPServer;nlp::NLPProcessor;nlp::NLPIndex;nlp::NLPToken]] + +## [[sign: nlp::NLPProcessor]] + +> [[doc: nlp::NLPProcessor]] + +[[uml: nlp | format: svg, mentities: nlp::nlp_server;nlp::nlp;nlp::nlp_index;nlp::stanford]] + +### [[sign: nlp::NLPJavaProcessor]] + +> [[doc: nlp::NLPJavaProcessor]] + +~~~nit +import nlp + +var proc = new NLPJavaProcessor("path/to/StanfordCoreNLP/jars") + +var doc = proc.process("String to analyze") + +for sentence in doc.sentences do + for token in sentence.tokens do + print "{token.lemma}: {token.pos}" + end +end +~~~ + +### [[sign: nlp::NLPServer]] + +> [[doc: nlp::NLPServer]] + +The [[nlp::NLPServer]] provides a wrapper around the StanfordCoreNLPServer. + +See `https://stanfordnlp.github.io/CoreNLP/corenlp-server.html`. + +~~~nit +import nlp + +var cp = "/path/to/StanfordCoreNLP/jars" +var srv = new NLPServer(cp, 9000) +srv.start +~~~ + +### NLPClient + +The [[nlp::NLPClient]] is used as a [[nlp::NLPProcessor]] with a NLPServer backend. + +~~~nit +import nlp + +var cli = new NLPClient("http://localhost:9000") +var doc = cli.process("String to analyze") +~~~ + +## NLPIndex + +[[nlp::NLPIndex]] extends the StringIndex to use a NLPProcessor to tokenize, lemmatize and +tag the terms of a document. + +~~~nit +import nlp + +var proc = new NLPJavaProcessor("path/to/StanfordCoreNLP/jars") +var index = new NLPIndex(proc) + +var d1 = index.index_string("Doc 1", "/uri/1", "this is a sample") +var d2 = index.index_string("Doc 2", "/uri/2", "this and this is another example") +assert index.documents.length == 2 + +var matches = index.match_string("this sample") +assert matches.first.document == d1 +~~~ + +## [[sign: nlp::NLPDocument]] + +> [[doc: nlp::NLPDocument]] + +## TODO + +* Use JWrapper +* Use options to choose CoreNLP analyzers +* Analyze sentences dependencies +* Analyze sentiment + +## Authors + +This project is maintained by [[ini-maintainer: nlp]]. diff --git a/lib/nlp/README.md b/lib/nlp/README.md index 3ed2c5ead5..59d5e1864a 100644 --- a/lib/nlp/README.md +++ b/lib/nlp/README.md @@ -1,4 +1,13 @@ -# Nit wrapper for Stanford CoreNLP +# `nlp` - Nit wrapper for Stanford CoreNLP + +* [`NLPProcessor`](#`NLPProcessor`) +* [`NLPJavaProcessor`](#`NLPJavaProcessor`) +* [`NLPServer`](#`NLPServer`) +* [NLPClient](#NLPClient) +* [NLPIndex](#NLPIndex) +* [`NLPDocument`](#`NLPDocument`) +* [TODO](#TODO) +* [Authors](#Authors) Stanford CoreNLP provides a set of natural language analysis tools which can take raw text input and give the base forms of words, their parts of speech, whether @@ -9,14 +18,34 @@ sentiment, etc. This wrapper needs the Stanford CoreNLP jars that run on Java 1.8+. -See http://nlp.stanford.edu/software/corenlp.shtml. +> See http://nlp.stanford.edu/software/corenlp.shtml. + +> * `NLPClient` - A NLPProcessor using a NLPServer as backend + +* `NLPDocument` - A `Document` represent a text input given to the NLP processor. +* `NLPFileIndex` - A FileIndex based using a NLPProcessor +* `NLPIndex` - A StringIndex using a NLPProcessor to parse and vectorize strings +* `NLPJavaProcessor` - Wrapper around StanfordNLP jar. +* `NLPProcessor` - Natural Language Processor +* `NLPSentence` - Represent one sentence in a `Document`. +* `NLPServer` - Stanford web server +* `NLPToken` - Represent one word (or puncutation mark) in a `NLPSentence`. + +## `NLPProcessor` -## NLPProcessor +> NLPProcessor provides natural language processing for input text and files. +> Analyzed documents can be manipulated through the resulting NLPDocument. -### Java client +![Diagram for `nlp`](uml-nlp-2.svg) -~~~nitish -var proc = new NLPProcessor("path/to/StanfordCoreNLP/jars") +### `NLPJavaProcessor` + +> FIXME this should use the Java FFI. + +~~~nit +import nlp + +var proc = new NLPJavaProcessor("path/to/StanfordCoreNLP/jars") var doc = proc.process("String to analyze") @@ -27,13 +56,20 @@ for sentence in doc.sentences do end ~~~ -### NLPServer +### `NLPServer` + +> Runs the server on `port`. -The NLPServer provides a wrapper around the StanfordCoreNLPServer. +For more details about the stanford NLP server see +https://stanfordnlp.github.io/CoreNLP/corenlp-server.html + +The [NLPServer](nlp::NLPServer) provides a wrapper around the StanfordCoreNLPServer. See `https://stanfordnlp.github.io/CoreNLP/corenlp-server.html`. -~~~nitish +~~~nit +import nlp + var cp = "/path/to/StanfordCoreNLP/jars" var srv = new NLPServer(cp, 9000) srv.start @@ -41,32 +77,45 @@ srv.start ### NLPClient -The NLPClient is used as a NLPProcessor with a NLPServer backend. +The [NLPClient](nlp::NLPClient) is used as a [NLPProcessor](nlp::NLPProcessor) with a NLPServer backend. + +~~~nit +import nlp -~~~nitish var cli = new NLPClient("http://localhost:9000") var doc = cli.process("String to analyze") ~~~ ## NLPIndex -NLPIndex extends the StringIndex to use a NLPProcessor to tokenize, lemmatize and +[NLPIndex](nlp::NLPIndex) extends the StringIndex to use a NLPProcessor to tokenize, lemmatize and tag the terms of a document. -~~~nitish +~~~nit +import nlp + +var proc = new NLPJavaProcessor("path/to/StanfordCoreNLP/jars") var index = new NLPIndex(proc) var d1 = index.index_string("Doc 1", "/uri/1", "this is a sample") var d2 = index.index_string("Doc 2", "/uri/2", "this and this is another example") assert index.documents.length == 2 -matches = index.match_string("this sample") +var matches = index.match_string("this sample") assert matches.first.document == d1 ~~~ +## `NLPDocument` + +> Once processed, it contains a list of sentences that contain tokens. + ## TODO * Use JWrapper * Use options to choose CoreNLP analyzers * Analyze sentences dependencies * Analyze sentiment + +## Authors + +This project is maintained by **Alexandre Terrasa **. diff --git a/lib/popcorn/README.docdown.md b/lib/popcorn/README.docdown.md new file mode 100644 index 0000000000..faf34169fc --- /dev/null +++ b/lib/popcorn/README.docdown.md @@ -0,0 +1,630 @@ +# `popcorn` - Web application framework for Nit + +[[toc: popcorn]] + +**Why endure plain corn when you can pop it?!** + +[[popcorn | text: Popcorn]] is a minimal yet powerful nit web application framework that provides cool +features for lazy developpers. + +Popcorn is built over nitcorn to provide a clean and user friendly interface +without all the boiler plate code. + +[[features: popcorn | mentities: popcorn::pop_sessions;popcorn::pop_tasks;popcorn::pop_json;popcorn::pop_templates;popcorn::pop_config;popcorn::popcorn;popcorn::pop_tests;popcorn::pop_routes;popcorn::pop_repos;popcorn::pop_auth;popcorn::pop_logging;popcorn::pop_tracker;popcorn::pop_handlers;popcorn::pop_validation]] + +## What does it taste like? + +[[uml: popcorn | format: svg, mentities: popcorn::pop_sessions;popcorn::pop_tasks;popcorn::pop_json;popcorn::pop_templates;popcorn::pop_config;popcorn::popcorn;popcorn::pop_tests;popcorn::pop_routes;popcorn::pop_repos;popcorn::pop_auth;popcorn::pop_logging;popcorn::pop_tracker;popcorn::pop_handlers;popcorn::pop_validation]] + +Set up is quick and easy as 10 lines of code. +Create a file `app.nit` and add the following code: + +[[code: popcorn::example_hello]] + +The Popcorn app listens on port 3000 for connections. +The app responds with "Hello World!" for requests to the root URL (`/`) or **route**. +For every other path, it will respond with a **404 Not Found**. + +The `req` (request) and `res` (response) parameters are the same that nitcorn provides +so you can do anything else you would do in your route without Popcorn involved. + +Run the app with the following command: + +~~~bash +$ nitc app.nit && ./app +~~~ + +Then, load [http://localhost:3000](http://localhost:3000) in a browser to see the output. + +Here the output using the `curl` command: + +~~~bash +$ curl localhost:3000 +

Hello World!

+ +$ curl localhost:3000/wrong_uri + + + + +Not Found + + +

404 Not Found

+ + +~~~ + +This is why we love popcorn! + +## Basic routing + +**Routing** refers to determining how an application responds to a client request +to a particular endpoint, which is a URI (or path) and a specific HTTP request +method GET, POST, PUT or DELETE (other methods are not suported yet). + +Each route can have one or more handler methods, which are executed when the route is matched. + +Route handlers definition takes the following form: + +~~~nitish +import popcorn + +class MyHandler + super Handler + + redef fun METHOD(req, res) do end +end +~~~ + +Where: + +* `MyHandler` is the name of the handler you will add to the app. +* `METHOD` can be replaced by `get`, `post`, `put` or `delete`. + +The following example responds to GET and POST requests: + +~~~ +import popcorn + +class MyHandler + super Handler + + redef fun get(req, res) do res.send "Got a GET request" + redef fun post(req, res) do res.send "Got a POST request" +end +~~~ + +To make your handler responds to a specific route, you have to add it to the app. + +Respond to POST request on the root route (`/`), the application's home page: + +~~~ +var app = new App +app.use("/", new MyHandler) +~~~ + +Respond to a request to the `/user` route: + +~~~ +app.use("/user", new MyHandler) +~~~ + +For more details about routing, see the routing section. + +## Serving static files with Popcorn + +To serve static files such as images, CSS files, and JavaScript files, use the +Popcorn built-in handler [[popcorn::StaticHandler | text: `StaticHandler`]]. + +Pass the name of the directory that contains the static assets to the StaticHandler +init method to start serving the files directly. +For example, use the following code to serve images, CSS files, and JavaScript files +in a directory named `public`: + +~~~ +app.use("/", new StaticHandler("public/")) +~~~ + +Now, you can load the files that are in the `public` directory: + +~~~raw +http://localhost:3000/images/trollface.jpg +http://localhost:3000/css/style.css +http://localhost:3000/js/app.js +http://localhost:3000/hello.html +~~~ + +Popcorn looks up the files relative to the static directory, so the name of the +static directory is not part of the URL. +To use multiple static assets directories, add the `StaticHandler` multiple times: + +~~~ +app.use("/", new StaticHandler("public/")) +app.use("/", new StaticHandler("files/")) +~~~ + +Popcorn looks up the files in the order in which you set the static directories +with the `use` method. + +To create a virtual path prefix (where the path does not actually exist in the file system) +for files that are served by the `StaticHandler`, specify a mount path for the +static directory, as shown below: + +~~~ +app.use("/static/", new StaticHandler("public/")) +~~~ + +Now, you can load the files that are in the public directory from the `/static` +path prefix. + +~~~raw +http://localhost:3000/static/images/trollface.jpg +http://localhost:3000/static/css/style.css +http://localhost:3000/static/js/app.js +http://localhost:3000/static/hello.html +~~~ + +However, the path that you provide to the `StaticHandler` is relative to the +directory from where you launch your app. +If you run the app from another directory, it’s safer to use the absolute path of +the directory that you want to serve. + +In some cases, you can want to redirect request to static files to a default file +instead of returning a 404 error. +This can be achieved by specifying a default file in the StaticHandler: + +~~~ +app.use("/static/", new StaticHandler("public/", "default.html")) +~~~ + +This way all non-matched queries to the StaticHandler will be answered with the +`default.html` file. + +## Advanced Routing + +**Routing** refers to the definition of application end points (URIs) and how +they respond to client requests. For an introduction to routing, see the Basic routing +section. + +The following code is an example of a very basic route. + +~~~ +import popcorn + +class HelloHandler + super Handler + + redef fun get(req, res) do res.send "Hello World!" +end + +var app = new App +app.use("/", new HelloHandler) +~~~ + +### Route methods + +A **route method** is derived from one of the HTTP methods, and is attached to an +instance of the [[popcorn::Handler]] class. + +The following code is an example of routes that are defined for the GET and the POST +methods to the root of the app. + +~~~ +import popcorn + +class GetPostHandler + super Handler + + redef fun get(req, res) do res.send "GET request to the homepage" + redef fun post(req, res) do res.send "POST request to the homepage" +end + +var app = new App +app.use("/", new GetPostHandler) +~~~ + +Popcorn supports the following routing methods that correspond to HTTP methods: +get, post, put, and delete. + +The request query string is accessed through the `req` parameter: + +[[code: popcorn::example_query_string]] + +Post parameters can also be accessed through the `req` parameter: + +[[code: popcorn::example_post_handler]] + +There is a special routing method, `all(res, req)`, which is not derived from any +HTTP method. This method is used to respond at a path for all request methods. + +In the following example, the handler will be executed for requests to "/user" +whether you are using GET, POST, PUT, DELETE, or any other HTTP request method. + +~~~ +import popcorn + +class AllHandler + super Handler + + redef fun all(req, res) do res.send "Every request to the homepage" +end +~~~ + +Using the `all` method you can also implement other HTTP request methods. + +~~~ +import popcorn + +class MergeHandler + super Handler + + redef fun all(req, res) do + if req.method == "MERGE" then + # handle that method + else super # keep handle GET, POST, PUT and DELETE methods + end +end +~~~ + +### Route paths + +**Route paths**, in combination with a request handlers, define the endpoints at +which requests can be made. +Route paths can be strings, parameterized strings or glob patterns. +Query strings such as `?q=foo`are not part of the route path. + +Popcorn uses the `AppRoute::match(uri)` method to match the route paths. + +Here are some examples of route paths based on strings. + +This route path will match requests to the root route, `/`. + +~~~ +import popcorn + +class MyHandler + super Handler + + redef fun get(req, res) do res.send "Got a GET request" +end + +var app = new App +app.use("/", new MyHandler) +~~~ + +This route path will match requests to `/about`. + +~~~ +app.use("/about", new MyHandler) +~~~ + +This route path will match requests to `/random.text`. + +~~~ +app.use("/random.text", new MyHandler) +~~~ + +During the query/response process, routes are matched by order of declaration +through the `App::use` method. + +The app declared in this example will try to match the routes in this order: + +1. `/` +2. `/about` +3. `/random.text` + +### Route parameters + +**Route parameters** are variable parts of a route path. They can be used to path +arguments within the URI. +Parameters in a route are prefixed with a colon `:` like in `:userId`, `:year`. + +The following example declares a handler `UserHome` that responds with the `user` +name. + +[[code: popcorn::example_param_route]] + +The `UserHome` handler listen to every path matching `/:user`. This can be `/Morriar`, +`/10`, ... but not `/Morriar/profile` since route follow the strict matching rule. + +### Glob routes + +**Glob routes** are routes that match only on a prefix, thus accepting a wider range +of URI. +Glob routes end with the symbol `*`. + +Here we define a `UserItem` handler that will respond to any URI matching the prefix +`/user/:user/item/:item`. +Note that glob route are compatible with route parameters. + +[[code: popcorn::example_glob_route]] + +Example from `popcorn::example_angular`: + +[[code: popcorn::example_angular]] + +## Response methods + +The methods on the response object (`res`), can is used to manipulate the +request-response cycle. +If none of these methods are called from a route handler, the client request will +receive a `404 Not found` error. + +* `res.html()` Send a HTML response. +* `res.json()` Send a JSON response. +* `res.redirect()` Redirect a request. +* `res.send()` Send a response of various types. +* `res.error()` Set the response status code and send its message as the response body. + +## Response cycle + +When the popcorn [[popcorn::App | text: `App`]] receives a request, the response cycle is the following: + +1. `pre-middlewares` lookup matching middlewares registered with `use_before(pre_middleware)`: + 1. execute matching middleware by registration order + 2. if a middleware send a response then let the `pre-middlewares` loop continue + with the next middleware +2. `response-handlers` lookup matching handlers registered with `use(handler)`: + 1. execute matching middleware by registration order + 2. if a middleware send a response then stop the `response-handlers` loop + 3. if no hander matches or sends a response, generate a 404 response +3. `post-middlewares` lookup matching handlers registered with `use_after(post_handler)`: + 1. execute matching middleware by registration order + 2. if a middleware send a response then let the `post-middlewares` loop continue + with the next middleware + +## Middlewares + +### Overview + +**Middleware** handlers are handlers that typically do not send `HttpResponse` responses. +Middleware handlers can perform the following tasks: + +* Execute any code. +* Make changes to the request and the response objects. +* End its action and pass to the next handler in the stack. + +If a middleware handler makes a call to `res.send()`, it provoques the end the +request-response cycle and the response is sent to the client. + +### Ultra simple logger example + +Here is an example of a simple “Hello World” Popcorn application. +We add a middleware handler to the application called MyLogger that prints a simple +log message in the app stdout. + +~~~ +import popcorn + +class MyLogger + super Handler + + redef fun all(req, res) do print "Request Logged!" +end + +class HelloHandler + super Handler + + redef fun get(req, res) do res.send "Hello World!" +end + + +var app = new App +app.use_before("/*", new MyLogger) +app.use("/", new HelloHandler) +app.listen("localhost", 3000) +~~~ + +By using the `MyLogger` handler to the route `/*` we ensure that every requests +(even 404 ones) pass through the middleware handler. +This handler just prints “Request Logged!” when a request is received. + +Be default, the order of middleware execution is that are loaded first are also executed first. +To ensure our middleware `MyLogger` will be executed before all the other, we add it +with the `use_before` method. + +### Ultra cool, more advanced logger example + +Next, we’ll create a middleware handler called “LogHandler” that prints the requested +uri, the response status and the time it took to Popcorn to process the request. + +This example gives a simplified version of the [[popcorn::RequestClock | text: `RequestClock`]] and [[popcorn::ConsoleLog | text: `ConsoleLog`]] middlewares. + +[[code: popcorn::example_advanced_logger]] + +First, we attach a new attribute `timer` to every `HttpRequest`. +Doing so we can access our data from all handlers that import our module, directly +from the `req` parameter. + +We use the new middleware called `RequestTimeHandler` to initialize the request timer. +Because of the `use_before` method, the `RequestTimeHandler` middleware will be executed +before all the others. + +We then let the `HelloHandler` produce the response. + +Finally, our `LogHandler` will display a bunch of data and use the request `timer` +to display the time it took to process the request. +Because of the `use_after` method, the `LogHandler` middleware will be executed after +all the others. + +The app now uses the `RequestTimeHandler` middleware for every requests received +by the Popcorn app. +The page is processed the `HelloHandler` to display the index page. +And, before every response is sent, the `LogHandler` is activated to display the +log line. + +Because you have access to the request object, the response object, and all the +Popcorn API, the possibilities with middleware functions are endless. + +### Built-in middlewares + +Starting with version 0.1, Popcorn provide a set of built-in middleware that can +be used to develop your app faster. + +* `RequestClock`: initializes requests clock. +* `ConsoleLog`: displays resquest and response status in console (can be used with `RequestClock`). +* `SessionInit`: initializes requests session (see the `Sessions` section). +* `StaticHandler`: serves static files (see the `Serving static files with Popcorn` section). +* `Router`: a mountable mini-app (see the `Mountable routers` section). + +## Mountable routers + +Use the [[popcorn::Router | text: `Router`]] class to create modular, mountable route handlers. +A Router instance is a complete middleware and routing system; for this reason, +it is often referred to as a “mini-app”. + +The following example creates a router as a module, loads a middleware handler in it, +defines some routes, and mounts the router module on a path in the main app. + +[[code: popcorn::example_router]] + +The app will now be able to handle requests to /user and /user/profile, as well +as call the `Time` middleware handler that is specific to the route. + +## Error handling + +**Error handling** is based on middleware handlers. + +Define error-handling middlewares in the same way as other middleware handlers: + +[[code: popcorn::example_simple_error_handler]] + +In this example, every non-200 response is caught by the `SimpleErrorHandler` +that print an error in stdout. + +By defining multiple middleware error handlers, you can take multiple action depending +on the kind of error or the kind of interface you provide (HTML, XML, JSON...). + +Here an example of the 404 custom error page in HTML: + +~~~ +import popcorn +import template + +class HtmlErrorTemplate + super Template + + var status: Int + var message: nullable String + + redef fun rendering do add """ + + + + + {{{message or else status}}} + + +

{{{status}}} {{{message or else ""}}}

+ + """ +end + +class HtmlErrorHandler + super Handler + + redef fun all(req, res) do + if res.status_code != 200 then + res.send(new HtmlErrorTemplate(res.status_code, "An error occurred!")) + end + end +end + +var app = new App +app.use("/*", new HtmlErrorHandler) +app.listen("localhost", 3000) +~~~ + +## [[sign: popcorn::pop_sessions]] + +> [[doc: popcorn::pop_sessions]] + +**Sessions** can be used thanks to the built-in [[popcorn::SessionInit | text: `SessionInit`]] middleware. + +Here a simple example of login button that define a value in the `req` session. + +[[code: popcorn::example_session]] + +Notice the use of the `SessionInit` on the `/*` route. You must use the +`SessionInit` first to initialize the request session. +Without that, your request session will be set to `null`. +If you don't use sessions in your app, you do not need to include that middleware. + +## Database integration + +### Mongo DB + +If you want to persist your data, Popcorn works well with MongoDB. + +In this example, we will show how to store and list user with a Mongo database. + +First let's define a handler that access the database to list all the user. +The mongo database reference is passed to the UserList handler through the `db` attribute. + +Then we define a handler that displays the user creation form on GET requests. +POST requests are used to save the user data. + +[[code: popcorn::example_mongodb]] + +## Angular.JS integration + +Loving [AngularJS](https://angularjs.org/)? Popcorn is made for Angular and for you! + +Using the StaticHandler with a glob route, you can easily redirect all HTTP requests +to your angular controller: + +[[code: popcorn::example_static_default]] + +Because the StaticHandler will not find the angular routes as static files, +you must specify the path to the default angular controller. +In this example, the StaticHandler will redirect any unknown requests to the `index.html` +angular controller. + +See the examples for a more detailed use case working with a JSON API. + +## [[sign: popcorn::pop_tasks]] + +> [[doc: popcorn::pop_tasks]] + +## [[sign: popcorn::pop_json]] + +> [[doc: popcorn::pop_json]] + +## [[sign: popcorn::pop_templates]] + +> [[doc: popcorn::pop_templates]] + +## [[sign: popcorn::pop_config]] + +> [[doc: popcorn::pop_config]] + +## [[sign: popcorn::pop_tests]] + +> [[doc: popcorn::pop_tests]] + +## [[sign: popcorn::pop_repos]] + +> [[doc: popcorn::pop_repos]] + +## [[sign: popcorn::pop_auth]] + +> [[doc: popcorn::pop_auth]] + +## [[sign: popcorn::pop_tracker]] + +> [[doc: popcorn::pop_tracker]] + +## [[sign: popcorn::pop_validation]] + +> [[doc: popcorn::pop_validation]] + +## Running the tests + +Run the nitunit automated tests with the following command: + +[[testing: popcorn]] + +## Authors + +This project is maintained by [[ini-maintainer: popcorn]]. diff --git a/lib/popcorn/README.md b/lib/popcorn/README.md index bbc9c95b0b..000cd72810 100644 --- a/lib/popcorn/README.md +++ b/lib/popcorn/README.md @@ -1,19 +1,70 @@ -# Popcorn +# `popcorn` - Web application framework for Nit + +* [What does it taste like?](#What-does-it-taste-like?) +* [Basic routing](#Basic-routing) +* [Serving static files with Popcorn](#Serving-static-files-with-Popcorn) +* [Advanced Routing](#Advanced-Routing) +* [Route methods](#Route-methods) +* [Route paths](#Route-paths) +* [Route parameters](#Route-parameters) +* [Glob routes](#Glob-routes) +* [Response methods](#Response-methods) +* [Response cycle](#Response-cycle) +* [Middlewares](#Middlewares) +* [Overview](#Overview) +* [Ultra simple logger example](#Ultra-simple-logger-example) +* [Ultra cool, more advanced logger example](#Ultra-cool,-more-advanced-logger-example) +* [Built-in middlewares](#Built-in-middlewares) +* [Mountable routers](#Mountable-routers) +* [Error handling](#Error-handling) +* [`pop_sessions`](#`pop_sessions`) +* [Database integration](#Database-integration) +* [Mongo DB](#Mongo-DB) +* [Angular.JS integration](#Angular.JS-integration) +* [`pop_tasks`](#`pop_tasks`) +* [`pop_json`](#`pop_json`) +* [`pop_templates`](#`pop_templates`) +* [`pop_config`](#`pop_config`) +* [`pop_tests`](#`pop_tests`) +* [`pop_repos`](#`pop_repos`) +* [`pop_auth`](#`pop_auth`) +* [`pop_tracker`](#`pop_tracker`) +* [`pop_validation`](#`pop_validation`) +* [Running the tests](#Running-the-tests) +* [Authors](#Authors) **Why endure plain corn when you can pop it?!** -Popcorn is a minimal yet powerful nit web application framework that provides cool +[Popcorn](popcorn) is a minimal yet powerful nit web application framework that provides cool features for lazy developpers. Popcorn is built over nitcorn to provide a clean and user friendly interface without all the boiler plate code. +* `pop_auth` - Authentification handlers. +* `pop_config` - Configuration file and options for Popcorn apps +* `pop_handlers` - Route handlers. +* `pop_json` - Introduce useful services for JSON REST API handlers. +* `pop_logging` +* `pop_repos` - Repositories for data management. +* `pop_routes` - Internal routes representation. +* `pop_sessions` - Session handlers +* `pop_tasks` - Popcorn threaded tasks +* `pop_templates` - Template rendering for popcorn +* `pop_tests` - Popcorn testing services +* `pop_tracker` - * `pop_validation` - Quick and easy validation framework for Json inputs +* `popcorn` - Application server abstraction on top of nitcorn. + ## What does it taste like? +![Diagram for `popcorn`](uml-popcorn-3.svg) + Set up is quick and easy as 10 lines of code. Create a file `app.nit` and add the following code: ~~~ +module example_hello is example + import popcorn class HelloHandler @@ -84,6 +135,7 @@ end ~~~ Where: + * `MyHandler` is the name of the handler you will add to the app. * `METHOD` can be replaced by `get`, `post`, `put` or `delete`. @@ -120,7 +172,7 @@ For more details about routing, see the routing section. ## Serving static files with Popcorn To serve static files such as images, CSS files, and JavaScript files, use the -Popcorn built-in handler `StaticHandler`. +Popcorn built-in handler [`StaticHandler`](popcorn::StaticHandler). Pass the name of the directory that contains the static assets to the StaticHandler init method to start serving the files directly. @@ -210,7 +262,7 @@ app.use("/", new HelloHandler) ### Route methods A **route method** is derived from one of the HTTP methods, and is attached to an -instance of the Handler class. +instance of the [Handler](popcorn::Handler) class. The following code is an example of routes that are defined for the GET and the POST methods to the root of the app. @@ -235,6 +287,8 @@ get, post, put, and delete. The request query string is accessed through the `req` parameter: ~~~ +module example_query_string is example + import popcorn import template @@ -260,6 +314,8 @@ app.listen("localhost", 3000) Post parameters can also be accessed through the `req` parameter: ~~~ +module example_post_handler is example + import popcorn import template @@ -321,7 +377,7 @@ which requests can be made. Route paths can be strings, parameterized strings or glob patterns. Query strings such as `?q=foo`are not part of the route path. -Popcorn uses the `Handler::match(uri)` method to match the route paths. +Popcorn uses the `AppRoute::match(uri)` method to match the route paths. Here are some examples of route paths based on strings. @@ -371,6 +427,8 @@ The following example declares a handler `UserHome` that responds with the `user name. ~~~ +module example_param_route is example + import popcorn class UserHome @@ -405,6 +463,8 @@ Here we define a `UserItem` handler that will respond to any URI matching the pr Note that glob route are compatible with route parameters. ~~~ +module example_glob_route is example + import popcorn class UserItem @@ -426,6 +486,43 @@ app.use("/user/:user/item/:item/*", new UserItem) app.listen("localhost", 3000) ~~~ +Example from `popcorn::example_angular`: + +~~~ +# This is an example of how to use angular.js with popcorn +module example_angular is example + +import popcorn +import popcorn::pop_json + +class CounterAPI + super Handler + + var counter = 0 + + fun json_counter: JsonObject do + var json = new JsonObject + json["label"] = "Visitors" + json["value"] = counter + return json + end + + redef fun get(req, res) do + res.json(json_counter) + end + + redef fun post(req, res) do + counter += 1 + res.json(json_counter) + end +end + +var app = new App +app.use("/counter", new CounterAPI) +app.use("/*", new StaticHandler("www/", "index.html")) +app.listen("localhost", 3000) +~~~ + ## Response methods The methods on the response object (`res`), can is used to manipulate the @@ -441,20 +538,20 @@ receive a `404 Not found` error. ## Response cycle -When the popcorn `App` receives a request, the response cycle is the following: +When the popcorn [`App`](popcorn::App) receives a request, the response cycle is the following: 1. `pre-middlewares` lookup matching middlewares registered with `use_before(pre_middleware)`: - 1. execute matching middleware by registration order - 2. if a middleware send a response then let the `pre-middlewares` loop continue - with the next middleware + 1. execute matching middleware by registration order + 2. if a middleware send a response then let the `pre-middlewares` loop continue + with the next middleware 2. `response-handlers` lookup matching handlers registered with `use(handler)`: - 1. execute matching middleware by registration order - 2. if a middleware send a response then stop the `response-handlers` loop - 3. if no hander matches or sends a response, generate a 404 response + 1. execute matching middleware by registration order + 2. if a middleware send a response then stop the `response-handlers` loop + 3. if no hander matches or sends a response, generate a 404 response 3. `post-middlewares` lookup matching handlers registered with `use_after(post_handler)`: - 1. execute matching middleware by registration order - 2. if a middleware send a response then let the `post-middlewares` loop continue - with the next middleware + 1. execute matching middleware by registration order + 2. if a middleware send a response then let the `post-middlewares` loop continue + with the next middleware ## Middlewares @@ -511,9 +608,11 @@ with the `use_before` method. Next, we’ll create a middleware handler called “LogHandler” that prints the requested uri, the response status and the time it took to Popcorn to process the request. -This example gives a simplified version of the `RequestClock` and `ConsoleLog` middlewares. +This example gives a simplified version of the [`RequestClock`](popcorn::RequestClock) and [`ConsoleLog`](popcorn::ConsoleLog) middlewares. ~~~ +module example_advanced_logger is example + import popcorn import realtime @@ -528,7 +627,7 @@ class RequestTimeHandler redef fun all(req, res) do req.timer = new Clock end -class LogHandler +class AdvancedLoggerHandler super Handler redef fun all(req, res) do @@ -541,7 +640,7 @@ class LogHandler end end -class HelloHandler +class AnotherHandler super Handler redef fun get(req, res) do res.send "Hello World!" @@ -549,8 +648,8 @@ end var app = new App app.use_before("/*", new RequestTimeHandler) -app.use("/", new HelloHandler) -app.use_after("/*", new LogHandler) +app.use("/", new AnotherHandler) +app.use_after("/*", new AdvancedLoggerHandler) app.listen("localhost", 3000) ~~~ @@ -586,12 +685,12 @@ be used to develop your app faster. * `RequestClock`: initializes requests clock. * `ConsoleLog`: displays resquest and response status in console (can be used with `RequestClock`). * `SessionInit`: initializes requests session (see the `Sessions` section). -* `StaticServer`: serves static files (see the `Serving static files with Popcorn` section). +* `StaticHandler`: serves static files (see the `Serving static files with Popcorn` section). * `Router`: a mountable mini-app (see the `Mountable routers` section). ## Mountable routers -Use the `Router` class to create modular, mountable route handlers. +Use the [`Router`](popcorn::Router) class to create modular, mountable route handlers. A Router instance is a complete middleware and routing system; for this reason, it is often referred to as a “mini-app”. @@ -599,6 +698,8 @@ The following example creates a router as a module, loads a middleware handler i defines some routes, and mounts the router module on a path in the main app. ~~~ +module example_router is example + import popcorn class AppHome @@ -613,7 +714,7 @@ class UserLogger redef fun all(req, res) do print "User logged" end -class UserHome +class UserHomepage super Handler redef fun get(req, res) do res.send "User Home" @@ -627,7 +728,7 @@ end var user_router = new Router user_router.use("/*", new UserLogger) -user_router.use("/", new UserHome) +user_router.use("/", new UserHomepage) user_router.use("/profile", new UserProfile) var app = new App @@ -646,6 +747,8 @@ as call the `Time` middleware handler that is specific to the route. Define error-handling middlewares in the same way as other middleware handlers: ~~~ +module example_simple_error_handler is example + import popcorn class SimpleErrorHandler @@ -653,19 +756,20 @@ class SimpleErrorHandler redef fun all(req, res) do if res.status_code != 200 then - print "An error occurred! {res.status_code})" + res.send("An error occurred!", res.status_code) end end end -class HelloHandler +class SomeHandler super Handler redef fun get(req, res) do res.send "Hello World!" end + var app = new App -app.use("/", new HelloHandler) +app.use("/", new SomeHandler) app.use("/*", new SimpleErrorHandler) app.listen("localhost", 3000) ~~~ @@ -716,13 +820,51 @@ app.use("/*", new HtmlErrorHandler) app.listen("localhost", 3000) ~~~ -## Sessions +## `pop_sessions` + +> Here a simple example on how to use sessions with popcorn: -**Sessions** can be used thanks to the built-in `SessionInit` middleware. +~~~ +import popcorn + +redef class Session + var is_logged = false +end + +class AppLogin + super Handler + + redef fun get(req, res) do + res.html """ +

Is logged: {{{req.session.as(not null).is_logged}}}

+
+ +
""" + end + + redef fun post(req, res) do + req.session.as(not null).is_logged = true + res.redirect("/") + end +end + +var app = new App +app.use_before("/*", new SessionInit) +app.use("/", new AppLogin) +app.listen("localhost", 3000) +~~~ + +Notice the use of the `SessionInit` on the `/*` route. You must use the +`SessionInit` first to initialize the request session. +Without that, your request session will be set to `null`. + +**Sessions** can be used thanks to the built-in [`SessionInit`](popcorn::SessionInit) middleware. Here a simple example of login button that define a value in the `req` session. ~~~ +module example_session is example + import popcorn redef class Session @@ -747,7 +889,7 @@ class AppLogin end var app = new App -app.use_before("/*", new SessionInit) +app.use("/*", new SessionInit) app.use("/", new AppLogin) app.listen("localhost", 3000) ~~~ @@ -772,6 +914,8 @@ Then we define a handler that displays the user creation form on GET requests. POST requests are used to save the user data. ~~~ +module example_mongodb is example + import popcorn import mongodb import template @@ -785,33 +929,26 @@ class UserList var users = db.collection("users").find_all(new JsonObject) var tpl = new Template - tpl.add "

Users

" - tpl.add "" + tpl.add """ +

Users

+ +

Add a new user

+ + + + + + +

All users

+
""" for user in users do tpl.add """ - - + + """ end tpl.add "
{{{user["login"] or else "null"}}}{{{user["password"] or else "null"}}}{{{user["login"] or else "null"}}}{{{user["password"] or else "null"}}}
" - res.html tpl - end -end - -class UserForm - super Handler - - var db: MongoDb - - redef fun get(req, res) do - var tpl = new Template - tpl.add """

Add a new user

-
- - - -
""" - res.html tpl + res.html(tpl) end redef fun post(req, res) do @@ -819,7 +956,7 @@ class UserForm json["login"] = req.post_args["login"] json["password"] = req.post_args["password"] db.collection("users").insert(json) - res.redirect "/" + res.redirect("/") end end @@ -828,7 +965,6 @@ var db = mongo.database("mongo_example") var app = new App app.use("/", new UserList(db)) -app.use("/new", new UserForm(db)) app.listen("localhost", 3000) ~~~ @@ -840,10 +976,12 @@ Using the StaticHandler with a glob route, you can easily redirect all HTTP requ to your angular controller: ~~~ +module example_static_default is example + import popcorn var app = new App -app.use("/*", new StaticHandler("my-ng-app/", "index.html")) +app.use("/", new StaticHandler("public/", "default.html")) app.listen("localhost", 3000) ~~~ @@ -853,3 +991,517 @@ In this example, the StaticHandler will redirect any unknown requests to the `in angular controller. See the examples for a more detailed use case working with a JSON API. + +## `pop_tasks` + +> Tasks allow you to execute code in another thread than the app listening loop. +> Useful when you want to run some tasks periodically. + +Let's say you want to purge the `downloads/` directory of your app every hour: + +~~~nitish +class PurgeTask + super PopTask + + var dir: String + + redef fun main do + loop + dir.rmdir + 3600.sleep + end + end +end + +var app = new App + +# Register a new task +app.register_task(new PurgeTask("downloads/")) + +# Add your handlers +# app.use('/', new MyHandler) + +# Run the tasks +app.run_tasks + +# Start the app +app.listen("0.0.0.0", 3000) +~~~ + +## `pop_json` + +> Validation and Deserialization of request bodies: + +~~~nit +class MyJsonHandler + super Handler + + # Validator used do validate the body + redef var validator = new MyFormValidator + + # Define the kind of objects expected by the deserialization process + redef type BODY: MyForm + + redef fun post(req, res) do + var post = validate_body(req, res) + if post == null then return # Validation error: let popcorn return a HTTP 400 + var form = deserialize_body(req, res) + if form == null then return # Deserialization error: let popcorn return a HTTP 400 + + # TODO do something with the input + print form.name + end +end + +class MyForm + serialize + + var name: String +end + +class MyFormValidator + super ObjectValidator + + init do + add new StringField("name", min_size=1, max_size=255) + end +end +~~~ + +## `pop_templates` + +> ## Basic templates + +Use TemplateString to render really basic templates that just need macro +replacements. + +Example: + +~~~nit +class TemplateStringHandler + super Handler + + redef fun get(req, res) do + # Values to add in the template + var values = new HashMap[String, String] + values["USER"] = "Morriar" + values["MYSITE"] = "My super website" + + # Render it with a shortcut + res.template_string(""" +

Hello %USER%!

+

Welcome to %MYSITE%.

+ """, values) + end +end +~~~ + +For larger templates, you can also use external files (makes your Nit code cleaner): + +~~~nit +class TemplateFileHandler + super Handler + + redef fun get(req, res) do + # Values to add in the template + var values = new HashMap[String, String] + values["USER"] = "Morriar" + values["MYSITE"] = "My super website" + + # Render it from an external file + res.template_file("example_template.tpl", values) + end +end +~~~ + +## Using pug templates + +Pug is a templating format provided by the external command `pug`. +For complex templates that need conditional or loop statements, pug can be a solution. + +See the pug syntax here: https://pugjs.org/api/getting-started.html + +~~~nit +class PugFileHandler + super Handler + + redef fun get(req, res) do + # Values to add in the template + var json = new JsonObject + json["user"] = "Morriar" + json["mysite"] = "My super website" + + # Render it from an external file + res.pug_file("example_template.pug", json) + end +end +~~~ + +## `pop_config` + +> `pop_config` provide a configuration framework for Popcorn apps based on ini +> files. + +By default `AppConfig` provides `app.host` and `app.port` keys, it's all we +need to start an app: + +~~~ +import popcorn +import popcorn::pop_config + +# Build config from options +var config = new AppConfig +config.parse_options(args) + +# Use options +var app = new App +app.listen(config.app_host, config.app_port) +~~~ + +For more advanced uses, `AppConfig` and `AppOptions` can be specialized to +offer additional config options: + +~~~ +import popcorn +import popcorn::pop_config + +class MyConfig + super AppConfig + + # My secret code I don't want to share in my source repository + fun secret: String do return opt_secret.value or else ini["secret"] or else "my-secret" + + # opt --secret + var opt_secret = new OptionString("My secret string", "--secret") + + redef init do + super + add_option opt_secret + end +end + +class SecretHandler + super Handler + + # Config to use to access `secret` + var config: MyConfig + + redef fun get(req, res) do + res.send config.secret + end +end + +var config = new MyConfig +config.parse_options(args) + +var app = new App +app.use("/secret", new SecretHandler(config)) +app.listen(config.app_host, config.app_port) +~~~ + +## `pop_tests` + +> ## Blackbox testing + +Popcorn allows you to test your apps using nitunit blackbox testing. + +With blackbox testing you compare the output of your program with a result file. + +To get started with blackbox testing, create a nitunit test suite and imports +the `pop_tests` module. + +You then need to build the app that will be tested by nitunit as shown in the +`TestExampleHello::test_example_hello` method. +Calling `run_test` will automatically set the `TestPopcorn::host` and `TestPopcorn::port` used for testing. + +Redefine the `client_test` method to write your scenario. +Here we use `curl` to access some URI on the app. + +~~~nitish +module test_example_hello is test + +import pop_tests +import example_hello + +class TestExampleHello + super TestPopcorn + test + + fun test_example_hello is test do + var app = new App + app.use("/", new HelloHandler) + run_test(app) + end + + redef fun client_test do + system "curl -s {host}:{port}" + system "curl -s {host}:{port}/" + system "curl -s {host}:{port}///////////" + system "curl -s {host}:{port}/not_found" + system "curl -s {host}:{port}/not_found/not_found" + end +end +~~~ + +The blackbox testing needs a reference result file against wich the test output +will be compared. +Create your expected result file in `test_example_hello.sav/test_example_hello.res`. + +Test your app by running nitunit: + +~~~bash +nitunit ./example_hello.nit +~~~ + +See `examples/hello_world` for the complete example. + +## `pop_repos` + +> Repositories are used to apply persistence on instances (or **documents**). +> Using repositories one can store and retrieve instance in a clean and maintenable +> way. + +This module provides the base interface `Repository` that defines the persistence +services available in all kind of repos. +`JsonRepository` factorizes all repositories dedicated to Json data or objects +serializable to Json. + +`MongoRepository` is provided as a concrete example of repository. +It implements all the services from `Repository` using a Mongo database as backend. + +Repositories can be used in Popcorn app to manage your data persistence. +Here an example with a book management app: + +~~~ +# First we declare the `Book` class. It has to be serializable so it can be used +# within a `Repository`. + +import popcorn +import popcorn::pop_repos +import popcorn::pop_json + +# Serializable book representation. +class Book + serialize + + # Book uniq ID + var id: String = (new MongoObjectId).id is serialize_as "_id" + + # Book title + var title: String + + # ... Other fields + + redef fun to_s do return title + redef fun ==(o) do return o isa SELF and id == o.id + redef fun hash do return id.hash +end + +# We then need to subclass the `MongoRepository` to provide Book specific services. + +# Book repository for Mongo +class BookRepo + super MongoRepository[Book] + + # Find books by title + fun find_by_title(title: String): Array[Book] do + var q = new JsonObject + q["title"] = title + return find_all(q) + end +end + +# The repository can be used in a Handler to manage book in a REST API. + +class BookHandler + super Handler + + var repo: BookRepo + + # Return a json array of all books + # + # If the get parameters `title` is provided, returns a json array of books + # matching the `title`. + redef fun get(req, res) do + var title = req.string_arg("title") + if title == null then + res.json new JsonArray.from(repo.find_all) + else + res.json new JsonArray.from(repo.find_by_title(title)) + end + end + + # Insert a new Book + redef fun post(req, res) do + var title = req.string_arg("title") + if title == null then + res.error 400 + return + end + var book = new Book(title) + repo.save book + res.json book + end +end + +# Let's wrap it all together in a Popcorn app: + +# Init database +var mongo = new MongoClient("mongodb://localhost:27017/") +var db = mongo.database("tests_app_{100000.rand}") +var coll = db.collection("books") + +# Init app +var app = new App +var repo = new BookRepo(coll) +app.use("/books", new BookHandler(repo)) +app.listen("localhost", 3000) +~~~ + +## `pop_auth` + +> For now, only Github OAuth is provided. + +See https://developer.github.com/v3/oauth/. + +This module provide 4 base classes that can be used together to implement a +Github OAuth handshake. + +Here an example of application using the Github Auth as login mechanism. + +There is 4 available routes: + +* `/login`: redirects the user to the Github OAuth login page (see `GithubLogin`) +* `/profile`: shows the currently logged in user (see `Profile Handler`) +* `/logout`: logs out the user by destroying the entry from the session (see `GithubLogout`) +* `/oauth`: callback url for Github service after player login (see `GithubOAuthCallBack`) + +Routes redirection are handled at the OAuth service registration. Please see +https://developer.github.com/v3/oauth/#redirect-urls for more niformation on how +to configure your service to provide smouth redirections beween your routes. + +~~~ +import popcorn +import popcorn::pop_auth + +class ProfileHandler + super Handler + + redef fun get(req, res) do + var session = req.session + if session == null then + res.send "No session :(" + return + end + var user = session.user + if user == null then + res.send "Not logged in" + return + end + res.send "

Hello {user.login}

" + end +end + +var client_id = "github client id" +var client_secret = "github client secret" + +var app = new App +app.use("/*", new SessionInit) +app.use("/login", new GithubLogin(client_id)) +app.use("/oauth", new GithubOAuthCallBack(client_id, client_secret)) +app.use("/logout", new GithubLogout) +app.use("/profile", new ProfileHandler) +app.listen("localhost", 3000) +~~~ + +Optionaly, you can use the `GithubUser` handler to provide access to the +Github user stored in session: + +~~~ +app.use("/api/user", new GithubUser) +~~~ + +## `pop_tracker` + +> ~~~nitish +~~~ + +app.use_after("/api/*", new PopTracker(config)) +app.use_after("/admin/*", new PopTracker(config)) + +~~~ + +To retrieve your tracker data use the `PopTrackerAPI` which serves the tracker +data in JSON format. + +~~~nitish +app.use("/api/tracker_data", new PopTrackerAPI(config)) +~~~ + +## `pop_validation` + +> Validators can be used in Popcorn apps to valid your json inputs before +> data processing and persistence. + +Here an example with a Book management app. We use an ObjectValidator to validate +the books passed to the API in the `POST /books` handler. + +~~~ +import popcorn +import popcorn::pop_json +import serialization + +# Serializable book representation. +class Book + super Serializable + + # Book ISBN + var isbn: String + + # Book title + var title: String + + # Book image (optional) + var img: nullable String + + # Book price + var price: Float +end + +class BookValidator + super ObjectValidator + + redef init do + add new ISBNField("isbn") + add new StringField("title", min_size=1, max_size=255) + add new StringField("img", required=false) + add new FloatField("price", min=0.0, max=999.0) + end +end + +class BookHandler + super Handler + + # Insert a new Book + redef fun post(req, res) do + var validator = new BookValidator + if not validator.validate(req.body) then + res.json(validator.validation, 400) + return + end + # TODO data persistence + end +end +~~~ + +## Running the tests + +Run the nitunit automated tests with the following command: + +~~~bash +nitunit . +~~~ + +## Authors + +This project is maintained by **Alexandre Terrasa **. diff --git a/lib/posix/README.docdown.md b/lib/posix/README.docdown.md new file mode 100644 index 0000000000..44cc84c5c3 --- /dev/null +++ b/lib/posix/README.docdown.md @@ -0,0 +1,15 @@ +# `posix` - Services conforming to POSIX + +The core module [[posix::posix | text: `posix`]] includes [[posix | text: POSIX]] services available on all POSIX compliant systems. +For services provided by some implementations of POSIX but absent from any POSIX version, +import [[posix::ext | text: `posix::ext`]]. + +[[uml: posix | format: svg, mentities: posix::ext;posix::posix]] + +[[features: posix | mentities: posix::ext;posix::posix]] + +[[features: posix | mentities: posix::Group;posix::Passwd]] + +## Authors + +This project is maintained by [[ini-maintainer: posix]]. diff --git a/lib/posix/README.md b/lib/posix/README.md index 3b73f9c7bc..0c31c6074b 100644 --- a/lib/posix/README.md +++ b/lib/posix/README.md @@ -1,5 +1,19 @@ -Services conforming to POSIX +# `posix` - Services conforming to POSIX -The core module `posix` includes POSIX services available on all POSIX compliant systems. +The core module [`posix`](posix::posix) includes [POSIX](posix) services available on all POSIX compliant systems. For services provided by some implementations of POSIX but absent from any POSIX version, -import `posix::ext`. +import [`posix::ext`](posix::ext). + +![Diagram for `posix`](uml-posix.svg) + +* `ext` - Services not defined in POSIX but provided by most implementations + +* `posix` - Services conforming to POSIX + +* `Group` - Information on a user group + +* `Passwd` - Information on a user account + +## Authors + +This project is maintained by **Alexis Laferrière **. diff --git a/lib/pthreads/README.docdown.md b/lib/pthreads/README.docdown.md new file mode 100644 index 0000000000..5070006de3 --- /dev/null +++ b/lib/pthreads/README.docdown.md @@ -0,0 +1,70 @@ +# `pthreads` - POSIX Threads support + +[[toc: pthreads]] + +The threads can be manipulated and synchronized using the classes [[pthreads::Thread | text: `Thread`]], +[[pthreads::Mutex | text: `Mutex`]] and [[pthreads::Barrier | text: `Barrier`]]. + +This group also provides two optional modules with thread-safe collections: + +* `redef_collections` redefines existing collection to make them thread-safe. + This incures a small overhead in all usage of the redefined collections. +* `concurrent_collections` intro new thread-safe collections. + +Theses services are implemented using the POSIX threads. + +You can also use the `is threaded` annotation on methods, which makes them run on their own thread. +Methods with self calls are not supported. + +A method or function annotated with `is threaded` has its return value changed during compilation. +You will get a subclass of `Thread`, even if there wasn't a return value before. You can know if the threaded method is done with the `is_done` boolean from `Thread`. +A call to the `join` method will block the execution until the threaded method is done, or immediatly return if it's already done. +`join` will return an object typed with the orginial return type, or `null` if there wasn't. + +Example from `pthreads::concurrent_array_and_barrier`: + +[[code: pthreads::concurrent_array_and_barrier]] + +[[features: pthreads | mentities: pthreads::extra;pthreads::threadpool;pthreads::pthreads;pthreads::redef_collections;pthreads::concurrent_collections]] + +## [[sign: pthreads::redef_collections]] + +> [[doc: pthreads::redef_collections]] + +## [[sign: pthreads::concurrent_collections]] + +> [[doc: pthreads::concurrent_collections]] + +## [[sign: pthreads::BlockingQueue]] + +> [[doc: pthreads::BlockingQueue]] + +## [[sign: pthreads::Barrier]] + +> [[doc: pthreads::Barrier]] + +## [[sign: pthreads::Mutex]] + +> [[doc: pthreads::Mutex]] + +## [[sign: pthreads::Thread]] + +> [[doc: pthreads::Thread]] + +## Known limitations: + +* Most services from the Nit library are not thread-safe. You must manage + your own mutex to avoid conflicts on shared data. +* FFI's global references are not thread-safe. + +## For more information: + +[[uml: pthreads | format: svg, mentities: pthreads::extra;pthreads::threadpool;pthreads::pthreads;pthreads::redef_collections;pthreads::concurrent_collections]] + +* See: `man pthreads` +* See: `examples/concurrent_array_and_barrier.nit` +* See: `examples/threaded_example.nit` + +## Authors + +This project is maintained by [[ini-maintainer: pthreads]]. diff --git a/lib/pthreads/README.md b/lib/pthreads/README.md index 44f3754de0..85d8eb6ffd 100644 --- a/lib/pthreads/README.md +++ b/lib/pthreads/README.md @@ -1,7 +1,17 @@ -# POSIX Threads support +# `pthreads` - POSIX Threads support -The threads can be manipulated and synchronized using the classes `Thread`, -`Mutex` and `Barrier`. +* [`redef_collections`](#`redef_collections`) +* [`concurrent_collections`](#`concurrent_collections`) +* [`BlockingQueue`](#`BlockingQueue`) +* [`Barrier`](#`Barrier`) +* [`Mutex`](#`Mutex`) +* [`Thread`](#`Thread`) +* [Known limitations:](#Known-limitations:) +* [For more information:](#For-more-information:) +* [Authors](#Authors) + +The threads can be manipulated and synchronized using the classes [`Thread`](pthreads::Thread), +[`Mutex`](pthreads::Mutex) and [`Barrier`](pthreads::Barrier). This group also provides two optional modules with thread-safe collections: @@ -19,6 +29,126 @@ You will get a subclass of `Thread`, even if there wasn't a return value before. A call to the `join` method will block the execution until the threaded method is done, or immediatly return if it's already done. `join` will return an object typed with the orginial return type, or `null` if there wasn't. +Example from `pthreads::concurrent_array_and_barrier`: + +~~~ +# A basic usage example of the modules `pthreads` and `pthreads::cocurrent_collections` +# +# 20 threads share an array and a barrier. They each insert 1000 strings into +# the array and wait at a barrier before finishing. +module concurrent_array_and_barrier is example + +import pthreads::concurrent_collections + +private class MyThread + super Thread + + # This `ConcurrentArray` has its own `Mutex` + var array: ConcurrentArray[String] + + # Use an explicit `Barrier` + var barrier: Barrier + + var id: Int + + redef fun main + do + # Print and add to Array 1000 times + for i in 1000.times do + var str = "thread {id}: {i}" + array.add str + end + + # Wait at the `barrier` + barrier.wait + + return id + end +end + +var n_threads = 20 + +# This `ConcurrentArray` has its own `Mutex` +var array = new ConcurrentArray[String] + +# Use an explicit `Barrier` +var barrier = new Barrier(n_threads) + +# Create all our threads +var threads = new Array[Thread] +for t in n_threads.times do + var thread = new MyThread(array, barrier, t) + threads.add thread + thread.start +end + +# Wait for the threads to complete +for thread in threads do + print "Thread {thread.join or else "null"} is done" +end + +print "{array.length} strings inserted" +~~~ + +* `concurrent_collections` - Introduces thread-safe concurrent collections +* `extra` - Offers some POSIX threads services that are not available on all platforms +* `pthreads` - Main POSIX threads support and intro the classes `Thread`, `Mutex` and `Barrier` +* `redef_collections` - Redef _some_ basic collections to be thread-safe +* `threadpool` - Introduces a minimal ThreadPool implementation using Tasks + +## `redef_collections` + +> This modules is intended to be used with scripts or quick prototypes. +> It makes thread safe _all_ instances of _some_ collections which +> also slightly slow down single threaded use. For more robust software, +> it is recommended to use `threads::concurrent_collections`. + +Thread-safe collections: + +- [x] `Array` +- [ ] `List` +- [ ] `HashMap` +- [ ] `HashSet` +- [ ] `Ref` +- [ ] `Queue` + +## `concurrent_collections` + +> This module offers new thread-safe collections. If you want to +> render basic collections thread-safe and don't mind the overhead cost, +> you can use `threads::redef_collections`. + +Concurrent collections: + +- [x] `ConcurrentArray` +- [x] `ConcurrentList` +- [ ] `ConcurrentHashMap` +- [ ] `ConcurrentHashSet` +- [ ] `ConcurrentRef` +- [ ] `ConcurrentQueue` + +Introduced collections specialize their critical methods according to the +current implementation in the standard library. If additional services +are added to the underlying collections by refinement or evolution, they +might need to be covered in the concurrent version. + +## `BlockingQueue` + + +## `Barrier` + +> Ensures that `count` threads call and block on `wait` before releasing them. + +## `Mutex` + +> Instances of this class can only be acquired by a single thread at any one +> point in time. Uses the recursive protocol so they can be locked many time by +> the same thread, must then be unlocked as many time. + +## `Thread` + +> Instances of this class are each used to launch and control a thread. + ## Known limitations: * Most services from the Nit library are not thread-safe. You must manage @@ -27,6 +157,12 @@ A call to the `join` method will block the execution until the threaded method i ## For more information: +![Diagram for `pthreads`](uml-pthreads-2.svg) + * See: `man pthreads` * See: `examples/concurrent_array_and_barrier.nit` -* See: ̀ examples/threaded_example.nit` +* See: `examples/threaded_example.nit` + +## Authors + +This project is maintained by **Alexis Laferrière **. diff --git a/lib/sdl2/README.docdown.md b/lib/sdl2/README.docdown.md new file mode 100644 index 0000000000..a1bf5ba1a1 --- /dev/null +++ b/lib/sdl2/README.docdown.md @@ -0,0 +1,47 @@ +# Low-level wrapper of the SDL 2.0 library (as `sdl2`) and SDL_image 2.0 (as `sdl2::image`) + +[[toc: sdl2]] + +The main entry point of this project, [[sdl2 | text: `sdl2`]], exposes some features of the base +library: video, events, syswm, etc. The alternative entry point `sdl2::image` offers +mainly `SDLSurface::load` to load images from PNG, JPG or TIF files. + +You can also import `sdl2::all` to get `sdl2` and all its sister libraries, which is only +`sdl2::image` at this point. + +Example from `sdl2::minimal`: + +[[code: sdl2::minimal]] + +[[uml: sdl2 | format: svg, mentities: sdl2::all;sdl2::sdl2;sdl2::syswm;sdl2::image;sdl2::mixer;sdl2::events;sdl2::sdl2_base]] + +[[features: sdl2 | mentities: sdl2::all;sdl2::sdl2;sdl2::syswm;sdl2::image;sdl2::mixer;sdl2::events;sdl2::sdl2_base]] + +## [[sign: sdl2::image]] + +> [[doc: sdl2::image]] + +## [[sign: sdl2::mixer]] + +> [[doc: sdl2::mixer]] + +## [[sign: sdl2::SDLSysWMInfo]] + +> [[doc: sdl2::SDLSysWMInfo]] + +## [[sign: sdl2::SDLMouseButtonEvent]] + +> [[doc: sdl2::SDLMouseButtonEvent]] + +## [[sign: sdl2::SDLEventBuffer]] + +> [[doc: sdl2::SDLEventBuffer]] + +## Examples + +See the `minimal` example within this project at `examples/minimal` for a simple example +of how to use this project. + +## Authors + +This project is maintained by [[ini-maintainer: sdl2]]. diff --git a/lib/sdl2/README.md b/lib/sdl2/README.md index 53bf50646e..09469bb8f3 100644 --- a/lib/sdl2/README.md +++ b/lib/sdl2/README.md @@ -1,13 +1,166 @@ -This is a low-level wrapper of the SDL 2.0 library (as `sdl2`) and SDL_image 2.0 (as `sdl2::image`). +# Low-level wrapper of the SDL 2.0 library (as `sdl2`) and SDL_image 2.0 (as `sdl2::image`) -The main entry point of this project, `sdl2`, exposes some features of the base +* [`image`](#`image`) +* [`mixer`](#`mixer`) +* [`SDLSysWMInfo`](#`SDLSysWMInfo`) +* [`SDLMouseButtonEvent`](#`SDLMouseButtonEvent`) +* [`SDLEventBuffer`](#`SDLEventBuffer`) +* [Examples](#Examples) +* [Authors](#Authors) + +The main entry point of this project, [`sdl2`](sdl2), exposes some features of the base library: video, events, syswm, etc. The alternative entry point `sdl2::image` offers mainly `SDLSurface::load` to load images from PNG, JPG or TIF files. You can also import `sdl2::all` to get `sdl2` and all its sister libraries, which is only `sdl2::image` at this point. -# Examples +Example from `sdl2::minimal`: + +~~~ +# An example to test and demonstrate the `sdl2` lib with `image` and `events` +module minimal is example + +import sdl2::all + +# Check for an error, then print and clear it +# +# Some errors are not fatal, so we ignore them at this level. +fun check_error(loc: String) +do + var error = sys.sdl.error.to_s + if not error.is_empty then + print "at {loc}: {error}" + sys.sdl.clear_error + end +end + +# Init `sdl2`, including video and events +sys.sdl.initialize((new SDLInitFlags).video) +check_error "init" + +# Init `sdl2::image` +sdl.img.initialize((new SDLImgInitFlags).png) +check_error "img_init" + +# Create a window +var window = new SDLWindow("Window title!".to_cstring, 800, 600, (new SDLWindowFlags).opengl) +check_error "window" + +# Create a render, the suffested one +var renderer = new SDLRenderer(window, -1, (new SDLRendererFlags).accelerated) +check_error "renderer" + +# Load an image +var surface = new SDLSurface.load("assets/fighter.png".to_cstring) +check_error "surface" +assert not surface.address_is_null + +# Alternative code to load a BMP image without `sdl2::image` +# +# var surface = new SDLSurface.load_bmp("assets/fighter.bmp".to_cstring) + +# Set the window icon +window.icon = surface +check_error "icon" + +# Get a texture out of that surface +var texture = new SDLTexture.from_surface(renderer, surface) +check_error "texture" + +# Allocate memory for reusable objects +var event = new SDLEventBuffer.malloc +var src = new SDLRect.nil +var dst = new SDLRect(0, 0, 128, 128) + +# Set the colors we will be using +var white = new SDLColor(255, 255, 255, 255) +var green = new SDLColor( 0, 255, 0, 255) +var spacy = new SDLColor( 25, 25, 50, 255) + +loop + # Loop over events until we get a quit event + while event.poll_event do + var higher_event = event.to_event + if higher_event isa SDLQuitEvent then + break label out + else if higher_event isa SDLMouseButtonDownEvent then + # Update `dst` to be centered on the latest click + dst.x = higher_event.x - dst.w/2 + dst.y = higher_event.y - dst.h/2 + end + end + + # Clear the screen with a spacy color + renderer.draw_color = spacy + renderer.clear + + # Draw the target box for the following `copy` + renderer.draw_color = green + renderer.draw_rect dst + + # Copy a texture to the screen + renderer.draw_color = white + renderer.copy(texture, src, dst) + + # Copy the back buffer to the screen + renderer.present + + check_error "present" + + 33.delay +end label out + +# Free all resources +event.free +src.free +dst.free + +texture.free +surface.free + +window.destroy +sdl.img.quit +sys.sdl.quit +~~~ + + +* `all` - Unites the main `sdl2` module and its sister library `sdl2::image` +* `events` - SDL 2 events and related services +* `image` - Services of the SDL_image 2.0 library +* `mixer` - SDL2 mixer with sample/sounds and music +* `sdl2` - Simple DirectMedia Layer (SDL) 2.0 services for easy window creation and 2D drawing +* `sdl2_base` - Basic SDL 2 features +* `syswm` - Window manager related SDL 2 services +![Diagram for `sdl2`](uml-sdl2.svg) + +## `image` + +> Offers `SDLSurface::load` which supports more image formats than `sdl2::sdl2` +> alone: JPG, PNG, TIF, GIT, ICO and much more. + +## `mixer` + +> C API documentation: https://www.libsdl.org/projects/SDL_mixer/docs/SDL_mixer.html + +## `SDLSysWMInfo` + +> Created using `SDLWindow::vm_info` + +## `SDLMouseButtonEvent` + +> This could as well be an abstract class. All instances of `SDLMouseButtonEvent` +> is either a `SDLMouseButtonUpEvent` or a `SDLMouseButtonDownEvent`. + +## `SDLEventBuffer` + +> An instance of this class should be used to call `poll_event` and `to_event`. + +## Examples See the `minimal` example within this project at `examples/minimal` for a simple example of how to use this project. + +## Authors + +This project is maintained by **Alexis Laferrière **. diff --git a/lib/serialization/README.docdown.md b/lib/serialization/README.docdown.md new file mode 100644 index 0000000000..2e450b7e7c --- /dev/null +++ b/lib/serialization/README.docdown.md @@ -0,0 +1,366 @@ +# `serialization` - Abstract serialization services + +[[toc: serialization]] + +The [[serialization]] services are based on the `serialize` and the `noserialize` annotations, +the `Serializable` interface and the implementations of `Serializer` and `Deserializer`. + +## The `serialize` annotation + +[[uml: serialization | format: svg, mentities: serialization::serialization;serialization::engine_tools;serialization::safe;serialization::caching;serialization::serialization_core;serialization::inspect]] + +A class annotated with `serialize` identifies it as a subclass of Serializable and +triggers the generation of customized serialization and deserialization services. + +~~~ +import serialization + +# Simple serializable class identifying a human +class Person + serialize + + # First and last name + var name: String + + # Year of birth (`null` if unknown) + var birth: nullable Int + + redef fun ==(o) do return o isa SELF and name == o.name and birth == o.birth + redef fun hash do return name.hash +end +~~~ + +The `Person` class also defines `==` and `hash`, this is optional but we will use it to make an important point. +By definition of a serializable class, an instance can be serialized to a stream, then deserialized. +The deserialized instance will not be the same instance, but they should be equal. +So, in this case, we can compare both instances with `==` to test their equality. + +Some conditions applies to the classes that can be annotated as `serialize`. +All attributes of the class must be serializable, runtime errors will be +raised when trying to serialize non-serializable attributes. + +In the class `Person`, all attributes are typed with classes the standards library. +These common types are defined defined as serializable by this project. +The attributes could also be typed with user-defined `serialize` +classes or any other subclass of [[serialization::Serializable | text: `Serializable`]]. + +~~~ +import serialization + +# Simple serializable class identifying a human +class Person + serialize + + # First and last name + var name: String + + # Year of birth (`null` if unknown) + var birth: nullable Int + + redef fun ==(o) do return o isa SELF and name == o.name and birth == o.birth + redef fun hash do return name.hash +end + +# This `serialize` class is composed of two `serialize` attributes +class Partnership + serialize + + var partner_a: Person + var partner_b: Person + + redef fun ==(o) do return o isa SELF and partner_a == o.partner_a and partner_b == o.partner_b + redef fun hash do return partner_a.hash + 1024*partner_b.hash +end +~~~ + +### Scope of the `serialize` annotation + +`serialize` can annotate class definitions, modules and attributes: + +* The annotation on a class applies only to the class definition, + only attributes declared locally will be serialized. + However, each definition of a class (a refinement or specialization) can be annotated with `serialize`. + +* A module declaration annotated with `serialize` states that all its class definitions + and locally declared attributes are serializable. + + ~~~ + module shared_between_clients is serialize + import serialization + ~~~ + +* Attribute annotated with `serialize` states that it is to be serialized, when the rest of the class does not. + The class will become subclass to `Serializable` but its attributes are not to be serialized by default. + Only the attributes with the `serialize` annotation will be serialized. + + ~~~ + import serialization + + # Only serialize the `name` + class UserCredentials + var name: String is serialize + var avatar_path: String = "/somepath/"+name is lazy + end + ~~~ + + Example from `serialization::custom_serialization`: + +[[code: serialization::custom_serialization]] + +## The `noserialize` annotation + +The `noserialize` annotation mark an exception in a `serialize` module or class definition. + +* By default a module is `noserialize`. There is no need to declare it as such. + +* A class definition annotated with `noserialize` within a `serialize` module will not be made serializable. + +* A `noserialize` attribute within a class or module annotated with `serialize` will not serialize this attribute. + The class will still be made subclass of `Serializable` and it won't affect the other attributes. + The `noserialize` attribute will not be set at deserialization. + Usually, it will also be annotated with `lazy` to get its value by another mean after the object has been deserialized. + + ~~~ + import serialization + + # Once again, only serialize the `name` + class UserCredentials + serialize + + var name: String + var avatar_path: String = "/somepath/"+name is noserialize, lazy + end + ~~~ + +## The `serialize_as` annotation + +By default, an attribute is identified in the serialization format by its Nit name. +The `serialize_as` attribute changes this behavior and sets the name of an attribute in the serialization format. + +This annotation can be useful to change the name of an attribute to what is expected by a remote service. +Or to use identifiers in the serialization format that are reserved keywords in Nit (like `class` and `type`). + +~~~ +import serialization + +class UserCredentials + serialize + + # Rename to "username" in JSON for compatibility with remote service + var name: String is serialize_as "username" + + # Rename to a shorter "ap" for a smaller JSON file + var avatar_path: String = "/somepath/"+name is lazy, serialize_as "ap" +end +~~~ + +## Custom serializable classes + +The annotation `serialize` should be enough for most cases, +but in some cases you need more control over the serialization process. + +For more control, create a subclass to `Serializable` and redefine `core_serialize_to`. +This method should use `Serializer::serialize_attribute` to serialize its components. +`serialize_attribute` works as a dictionary and organize attributes with a key. + +You will also need to redefine `Deserializer::deserialize_class` to support this specific class. +The method should only act on known class names, and call super otherwise. + +### Example: the User class + +The following example cannot use the `serialize` annotations +because some of the arguments to the `User` class need special treatment: + +* The `name` attribute is perfectly normal, it can be serialized and deserialized + directly. + +* The `password` attribute must be encrypted before being serialized, + and unencrypted on deserialization. + +* The `avatar` attributes is kept as ASCII art in memory. + It could be serialized as such but it is cleaner to only + serialize the path to its source on the file system. + The data is reloaded on deserialization. + +For this customization, the following code snippet implements +two serialization services: `User::core_serialize_to` and +`Deserializer::deserialize_class`. + +~~~ +module user_credentials + +import serialization + +# User credentials for a website +class User + super Serializable + + # User name + var name: String + + # Clear text password + var password: String + + # User's avatar image as data blob + var avatar: Image + + redef fun core_serialize_to(serializer: Serializer) + do + # This is the normal serialization process + serializer.serialize_attribute("name", name) + + # Serialized an encrypted version of the password + # + # Obviously, `rot(13)` is not a good encrption + serializer.serialize_attribute("pass", password.rot(13)) + + # Do not serialize the image, only its path + serializer.serialize_attribute("avatar_path", avatar.path) + end +end + +redef class Deserializer + redef fun deserialize_class(name) + do + if name == "User" then + # Deserialize normally + var user = deserialize_attribute("name").as(String) + + # Decrypt password + var pass = deserialize_attribute("pass").as(String).rot(-13) + + # Deserialize the path and load the avatar from the file system + var avatar_path = deserialize_attribute("avatar_path").as(String) + var avatar = new Image(avatar_path) + + return new User(user, pass, avatar) + end + + return super + end +end + +redef class String + fun rot(s: Int): String do return self +end + +# An image loaded in memory as ASCII art +# +# Not really useful for this example, provided for consistency only. +class Image + # Path on the filesystem for `self` + var path: String + + # ASCII art composing this image + var ascii_art: String = path.to_path.read_all is lazy +end + +~~~ + +See the documentation of the module [[serialization::serialization | text: `serialization::serialization`]] for more +information on the services to redefine. + +## Serialization services + +The `serialize` annotation and the `Serializable` class are used on +classes specific to the business domain. +To write (and read) instances of these classes to a persistent format +you must use implementations of [[serialization::Serializer | text: `Serializer`]] and [[serialization::Deserializer | text: `Deserializer`]]. + +The main implementations of these services are `JsonSerializer` and `JsonDeserializer`, +from the `json_serialization` module. + +~~~nitish +mport json +import user_credentials + +# Data to be serialized and deserialized +var couple = new Partnership( + new Person("Alice", 1985, new Image("alice.png")), + new Person("Bob", null, new Image("bob.png"))) + +var path = "serialized_data.json" +var writer = new FileWriter(path) +var serializer = new JsonSerializer(writer) +serializer.serialize couple +writer.close + +var reader = new FileReader(path) +var deserializer = new JsonDeserializer(reader.to_s) +var deserialized_couple = deserializer.deserialize +reader.close + +assert couple == deserialize_couple +~~~ + +## Limitations and TODO + +The serialization has some limitations: + +* A limitation of the JSON parser prevents deserializing from files + with more than one object. + This could be improved in the future, but for now you should + serialize a single object to each files and use different instances of + serializer and deserializer each time. + +* The serialization uses only the short name of a class, not its qualified name. + This will cause problem when different classes using the same name. + This could be solved partially in the compiler and the library. + A special attention must be given to the consistency of the name across + the different programs sharing the serialized data. + +* The serialization support in the compiler need some help to + deal with generic types. A solution is to use `nitserial`, + the next section explores this subject. + +## Dealing with generic types + +One limitation of the serialization support in the compiler is with generic types. +For example, the `Array` class is generic and serializable. +However, the runtime types of Array instances are parameterized and are unknown to the compiler. +So the compiler won't support serializing instances of `Array[MySerializable]`. + +The tool `nitserial` solves this problem at the level of user modules. +It does so by parsing a Nit module, group or project to find all known +parameterized types of generic classes. +It will then generating a Nit module to handle deserialization of these types. + +Usage steps to serialize parameterized types: + +* Write your program, let's call it `my_prog.nit`, + it must use some parameterized serializable types. + Let's say that you use `Array[MySerializable]`. + +* Run nitserial using `nitserial my_prog.nit` to + generate the file `my_prog_serial.nit`. + +* Compile your program by mixing in the generated module with: + `nitc my_prog.nit -m my_prog_serial.nit` + +This was a simple example, in practical cases you may need +to use more than one generated file. +For example, on a client/server system, an instance can be created +server-side, serialized and the used client-side. +In this case, two files will be generated by nitserial, +one for the server and one for the client. +Both the files should be compiled with both the client and the server. + +## [[sign: serialization::serialization_core]] + +> [[doc: serialization::serialization_core]] + +## [[sign: serialization::DeserializerCache]] + +> [[doc: serialization::DeserializerCache]] + +## [[sign: serialization::SerializerCache]] + +> [[doc: serialization::SerializerCache]] + +## [[sign: serialization::Deserializer]] + +> [[doc: serialization::Deserializer]] + +## Authors + +This project is maintained by [[ini-maintainer: serialization]]. diff --git a/lib/serialization/README.md b/lib/serialization/README.md index 3f6ac6c063..014bb60493 100644 --- a/lib/serialization/README.md +++ b/lib/serialization/README.md @@ -1,10 +1,27 @@ -# Abstract serialization services - -The serialization services are based on the `serialize` and the `noserialize` annotations, +# `serialization` - Abstract serialization services + +* [The `serialize` annotation](#The-`serialize`-annotation) +* [Scope of the `serialize` annotation](#Scope-of-the-`serialize`-annotation) +* [The `noserialize` annotation](#The-`noserialize`-annotation) +* [The `serialize_as` annotation](#The-`serialize_as`-annotation) +* [Custom serializable classes](#Custom-serializable-classes) +* [Example: the User class](#Example:-the-User-class) +* [Serialization services](#Serialization-services) +* [Limitations and TODO](#Limitations-and-TODO) +* [Dealing with generic types](#Dealing-with-generic-types) +* [`serialization_core`](#`serialization_core`) +* [`DeserializerCache`](#`DeserializerCache`) +* [`SerializerCache`](#`SerializerCache`) +* [`Deserializer`](#`Deserializer`) +* [Authors](#Authors) + +The [serialization](serialization) services are based on the `serialize` and the `noserialize` annotations, the `Serializable` interface and the implementations of `Serializer` and `Deserializer`. ## The `serialize` annotation +![Diagram for `serialization`](uml-serialization.svg) + A class annotated with `serialize` identifies it as a subclass of Serializable and triggers the generation of customized serialization and deserialization services. @@ -38,9 +55,25 @@ raised when trying to serialize non-serializable attributes. In the class `Person`, all attributes are typed with classes the standards library. These common types are defined defined as serializable by this project. The attributes could also be typed with user-defined `serialize` -classes or any other subclass of `Serializable`. +classes or any other subclass of [`Serializable`](serialization::Serializable). ~~~ +import serialization + +# Simple serializable class identifying a human +class Person + serialize + + # First and last name + var name: String + + # Year of birth (`null` if unknown) + var birth: nullable Int + + redef fun ==(o) do return o isa SELF and name == o.name and birth == o.birth + redef fun hash do return name.hash +end + # This `serialize` class is composed of two `serialize` attributes class Partnership serialize @@ -66,6 +99,7 @@ end ~~~ module shared_between_clients is serialize + import serialization ~~~ * Attribute annotated with `serialize` states that it is to be serialized, when the rest of the class does not. @@ -73,6 +107,8 @@ end Only the attributes with the `serialize` annotation will be serialized. ~~~ + import serialization + # Only serialize the `name` class UserCredentials var name: String is serialize @@ -80,6 +116,152 @@ end end ~~~ + Example from `serialization::custom_serialization`: + +~~~ +# Example of an ad hoc serializer that is tailored to transform business specific objects into customized representation. +# +# In the following, we expose 3 business classes, `E`, `A` and `B`, and a specific business serializer `RestrictedSerializer`. +# The principle is that the custom serialization logic in enclosed into the `RestrictedSerializer` and that the +# standard serializer is unchanged. +# +# The additional behaviors exposed are: +# +# * replace a full business object (see `E::id`): +# instead of serializing an attribute, the custom serializer uses a different representation. +# * inject a phantom attribute (see `E::phantom`): +# when serializing, the custom serializer injects new attributes. +# * hide a normally serialized attribute (see `E::semi_private`): +# when serializing, the custom serializer hides some specific attributes. +# +# The advantage of the approach is that it is done programmatically so can be adapted to real complex use cases. +# Basically, this is half-way between the full automatic serialization and the full manual serialisation. +module custom_serialization is example + +import serialization +import json::serialization_write + +# The root class of the business objects. +# This factorizes most services and domain knowledge used by the `RestrictedSerializer` +# +# In real enterprise-level code, the various specific behaviors can be specified in more semantic classifiers. +abstract class E + serialize + + # The semantic business identifier. + # + # With the `RestrictedSerializer`, references to `E` objects will be replaced with `id`-based information. + # This avoid to duplicate or enlarge the information cross-call wise. + # + # A future API/REST call can then request the _missing_ object from its identifier. + var id: String + + # A phantom attribute to be serialized by the custom `RestrictedSerializer`. + # + # This can be used to inject constant or computed information that make little sense to have as a genuine attribute in + # the Nit model. + fun phantom: String do return "So Much Fun" + + # An attribute not to be serialized by the custom `RestrictedSerializer`. + # e.g. we want it on the DB but not in API/REST JSON messages + # + # Note that the annotation `noserialize` hides the attribute for all serializers. + # To hide the attribute only in the `RestrictedSerializer`, it will have to actively ignore it. + var semi_private = "secret" + + # Test method that serializes `self` and prints with the standard JsonSerializer + fun ser_json + do + var w = new StringWriter + var js = new JsonSerializer(w) + js.plain_json = true + js.serialize(self) + print w + end + + # Test method that serializes `self` and prints with the custom RestrictedJsonSerializer. + fun ser_json2 + do + var w = new StringWriter + var js = new RestrictedJsonSerializer(w) + js.plain_json = true + js.serialize(self) + print w + end +end + +# Extends Serializer and adds specific business behaviors when dealing with business objects. +# +# As with standard Nit, additional level of customization can be achieved by adding more double-dispatching :) +# We can thus choose to locate the specific behavior in the serializer, or the serializees. +class RestrictedSerializer + super Serializer + + # This method is called to generate the attributes of a serialized representation + redef fun serialize_core(value) + do + super + + if value isa E then + # Inject additional special domain-specific information + serialize_attribute("more-data", value.phantom) + end + end + + # This method is called when trying to serialize a specific attribute + redef fun serialize_attribute(name, value) + do + var recv = current_object + if recv isa E then + # do not serialize `E::semi_private` + if name == "semi_private" then return + end + + if value isa E then + # Do not serialize references to `E`. + # Just use a domain-specific value that make sense in the business logic. + serialize_attribute(name, "ID:" + value.id) + return + end + + super + end +end + +# Extends JsonSerializer and adds specific business behaviors when dealing with business objects. +class RestrictedJsonSerializer + super JsonSerializer + super RestrictedSerializer +end + +# A business object, with an integer information +class A + super E + serialize + + # A business information + var i: Int +end + +# A business object associated with an `A`. +class B + super E + serialize + + # A business association + var a: A +end + +# The business data to serialize +var a = new A("a", 1) +var b = new B("b", a) + +a.ser_json +a.ser_json2 +b.ser_json +b.ser_json2 +~~~ + ## The `noserialize` annotation The `noserialize` annotation mark an exception in a `serialize` module or class definition. @@ -94,6 +276,8 @@ The `noserialize` annotation mark an exception in a `serialize` module or class Usually, it will also be annotated with `lazy` to get its value by another mean after the object has been deserialized. ~~~ + import serialization + # Once again, only serialize the `name` class UserCredentials serialize @@ -112,6 +296,8 @@ This annotation can be useful to change the name of an attribute to what is expe Or to use identifiers in the serialization format that are reserved keywords in Nit (like `class` and `type`). ~~~ +import serialization + class UserCredentials serialize @@ -158,6 +344,8 @@ two serialization services: `User::core_serialize_to` and ~~~ module user_credentials +import serialization + # User credentials for a website class User super Serializable @@ -191,13 +379,13 @@ redef class Deserializer do if name == "User" then # Deserialize normally - var user = deserialize_attribute("name") + var user = deserialize_attribute("name").as(String) # Decrypt password - var pass = deserialize_attribute("pass").rot(-13) + var pass = deserialize_attribute("pass").as(String).rot(-13) # Deserialize the path and load the avatar from the file system - var avatar_path = deserialize_attribute("avatar_path") + var avatar_path = deserialize_attribute("avatar_path").as(String) var avatar = new Image(avatar_path) return new User(user, pass, avatar) @@ -207,6 +395,10 @@ redef class Deserializer end end +redef class String + fun rot(s: Int): String do return self +end + # An image loaded in memory as ASCII art # # Not really useful for this example, provided for consistency only. @@ -215,12 +407,12 @@ class Image var path: String # ASCII art composing this image - var ascii_art: String = path.read_all is lazy + var ascii_art: String = path.to_path.read_all is lazy end ~~~ -See the documentation of the module `serialization::serialization` for more +See the documentation of the module [`serialization::serialization`](serialization::serialization) for more information on the services to redefine. ## Serialization services @@ -228,13 +420,13 @@ information on the services to redefine. The `serialize` annotation and the `Serializable` class are used on classes specific to the business domain. To write (and read) instances of these classes to a persistent format -you must use implementations of `Serializer` and `Deserializer`. +you must use implementations of [`Serializer`](serialization::Serializer) and [`Deserializer`](serialization::Deserializer). The main implementations of these services are `JsonSerializer` and `JsonDeserializer`, from the `json_serialization` module. -~~~ -import json +~~~nitish +mport json import user_credentials # Data to be serialized and deserialized @@ -307,3 +499,48 @@ server-side, serialized and the used client-side. In this case, two files will be generated by nitserial, one for the server and one for the client. Both the files should be compiled with both the client and the server. + +## `serialization_core` + +> This module declares the `serialize` annotation to mark Nit classes as serializable. +> For an introduction to this service, refer to the documentation of the `serialization` group. +> This documentation provides more technical information on interesting entitie of this module. + +Interesting entities for end users of serializable classes: + +* Serialize an instance subclass of `Serializable` with either + `Serializer::serializable` and `Serializable::serialize`. +* Deserialize an object using `Deserializer::deserialize`. + The object type must the be checked with an `assert` or otherwise. + +Interesting entities to create custom serializable classes: + +* Subclass `Serializable` to declare a class as serializable and to customize + the serialization and deserialization behavior. +* Redefine `Serializable::core_serialize_to` to customize the serialization + of the receiver class. +* Redefine `Deserializer::deserialize_class` to customize the deserialization + of a specific class by name. + +Interesting entities for serialization format: + +* Subclass `Serializer` and `Deserializer` with custom serices. +* In `Serializer`, `serialize` and `serialize_reference` must be redefined. +* In `Deserializer`; `deserialize`, `deserialize_attribute and`notify_of_creation` must be redefined. + +## `DeserializerCache` + +> Used by `Deserializer` to find already deserialized objects by their reference. + +## `SerializerCache` + +> Used by `Serializer` to avoid duplicating objects, by serializing them once, +> then using a reference. + +## `Deserializer` + +> The main service is `deserialize`. + +## Authors + +This project is maintained by **Alexis Laferrière **. diff --git a/lib/vsm/README.docdown.md b/lib/vsm/README.docdown.md new file mode 100644 index 0000000000..08aba870eb --- /dev/null +++ b/lib/vsm/README.docdown.md @@ -0,0 +1,137 @@ +# `vsm` - Vector Space Model + +[[toc: vsm]] + +> [[doc: vsm::vsm]] + +The [[vsm | text: `vsm`]] package provides the following features: + +* Vector comparison with cosine similarity. +* Vector indexing and matching with tf * idf. +* File indexing and matching to free text queries. + +[[features: vsm | mentities: vsm::bm25;vsm::vsm]] + +[[features: vsm | mentities: vsm::IndexMatchSorter;vsm::IndexMatch;vsm::StringIndex;vsm::Vector;vsm::FileIndex;vsm::Document;vsm::VSMIndex]] + +## [[sign: vsm::Vector]] + +> [[doc: vsm::Vector]] + +[[uml: vsm | format: svg, mentities: vsm::IndexMatchSorter;vsm::IndexMatch;vsm::StringIndex;vsm::Vector;vsm::FileIndex;vsm::Document;vsm::VSMIndex]] + +With VSM, [[vsm::Document | text: documents]] are represented by a n-dimensions [[vsm::Vector | text: vector]]. +Each dimension represent an attribute of the document or object. + +For text document, the count of each term found in the document if often used to +build vectors. + +### Creating a vector + +~~~ +import vsm + +var vector = new Vector +vector["term1"] = 2.0 +vector["term2"] = 1.0 +assert vector["term1"] == 2.0 +assert vector["term2"] == 1.0 +assert vector.norm.is_approx(2.236, 0.001) +~~~ + +### Comparing vectors + +~~~ +import vsm + +var v1 = new Vector +v1["term1"] = 1.0 +v1["term2"] = 2.0 + +var v2 = new Vector +v2["term2"] = 1.0 +v2["term3"] = 3.0 + +var query = new Vector +query["term2"] = 1.0 + +var s1 = query.cosine_similarity(v1) +var s2 = query.cosine_similarity(v2) +assert s1 > s2 +~~~ + +## [[sign: vsm::VSMIndex]] + +[[vsm::VSMIndex]] is a Document index based on VSM. + +> [[doc: vsm::VSMIndex]] + +This represents a minimalistic search engine. + +~~~ +import vsm + +var index = new VSMIndex + +var v1 = new Vector +v1["term1"] = 1.0 +v1["term2"] = 2.0 + +var v2 = new Vector +v2["term2"] = 1.0 +v2["term3"] = 3.0 + +var query = new Vector +query["term2"] = 1.0 + +var d1 = new Document("Doc 1", "/uri/1", v1) +index.index_document(d1) + +var d2 = new Document("Doc 2", "/uri/2", v2) +index.index_document(d2) + +assert index.documents.length == 2 + +query = new Vector +query["term1"] = 1.0 + +var matches = index.match_vector(query) +assert matches.first.document == d1 +~~~ + +## [[sign: vsm::StringIndex]] + +The [[vsm::StringIndex]] provides usefull services to index and match strings. + +~~~ +import vsm + +var index = new StringIndex + +var d1 = index.index_string("Doc 1", "/uri/1", "this is a sample") +var d2 = index.index_string("Doc 2", "/uri/2", "this and this is another example") +assert index.documents.length == 2 + +var matches = index.match_string("this sample") +assert matches.first.document == d1 +~~~ + +Example from `vsm::example_vsm`: + +[[code: vsm::example_vsm]] + +## [[sign: vsm::FileIndex]] + +The [[vsm::FileIndex]] is a StringIndex able to index and retrieve files. + +~~~nit +import vsm + +var index = new FileIndex + +index.index_files(["/path/to/doc/1", "/path/to/doc/2"]) +~~~ + +## Authors + +This project is maintained by [[ini-maintainer: vsm]]. diff --git a/lib/vsm/README.md b/lib/vsm/README.md index ea501328fe..ec4c795ae1 100644 --- a/lib/vsm/README.md +++ b/lib/vsm/README.md @@ -1,20 +1,53 @@ -# Vector Space Model +# `vsm` - Vector Space Model -Vector Space Model (VSM) is an algebraic model for representing text documents -(and any objects, in general) as vectors of identifiers, such as, for example, -index terms. +* [`Vector`](#`Vector`) +* [Creating a vector](#Creating-a-vector) +* [Comparing vectors](#Comparing-vectors) +* [`VSMIndex`](#`VSMIndex`) +* [`StringIndex`](#`StringIndex`) +* [`FileIndex`](#`FileIndex`) +* [Authors](#Authors) + +> Vector Space Model (VSM) is an algebraic model for representing text documents +> (and any objects, in general) as vectors of identifiers, such as, for example, +> index terms. It is used in information filtering, information retrieval, indexing and relevancy rankings. -The `vsm` package provides the following features: +The [`vsm`](vsm) package provides the following features: + * Vector comparison with cosine similarity. + * Vector indexing and matching with tf * idf. + * File indexing and matching to free text queries. -## Vectors +* `bm25` + +* `vsm` - Vector Space Model + +* `Document` - A Document to add in a VSMIndex + +* `FileIndex` - A VSMIndex for file indexing and matching + +* `IndexMatch` - A match to a `request` in an `Index` + +* `IndexMatchSorter` - Sort matches by similarity + +* `StringIndex` - A VSMIndex for string indexing and matching -With VSM, documents are represented by a n-dimensions vector. +* `VSMIndex` - A Document index based on VSM + +* `Vector` - A n-dimensions vector + +## `Vector` + +> *n-dimensions* vectors are used to represent a text document or an object. + +![Diagram for `vsm`](uml-vsm.svg) + +With VSM, [documents](vsm::Document) are represented by a n-dimensions [vector](vsm::Vector). Each dimension represent an attribute of the document or object. For text document, the count of each term found in the document if often used to @@ -23,6 +56,8 @@ build vectors. ### Creating a vector ~~~ +import vsm + var vector = new Vector vector["term1"] = 2.0 vector["term2"] = 1.0 @@ -34,6 +69,8 @@ assert vector.norm.is_approx(2.236, 0.001) ### Comparing vectors ~~~ +import vsm + var v1 = new Vector v1["term1"] = 1.0 v1["term2"] = 2.0 @@ -50,18 +87,31 @@ var s2 = query.cosine_similarity(v2) assert s1 > s2 ~~~ -## VSMIndex +## `VSMIndex` -VSMIndex is a Document index based on VSM. +[VSMIndex](vsm::VSMIndex) is a Document index based on VSM. -Using VSMIndex you can index documents associated with their vector. -Documents can then be matched to query vectors. +> Using VSMIndex you can index documents associated with their vector. +> Documents can then be matched to query vectors. This represents a minimalistic search engine. ~~~ +import vsm + var index = new VSMIndex +var v1 = new Vector +v1["term1"] = 1.0 +v1["term2"] = 2.0 + +var v2 = new Vector +v2["term2"] = 1.0 +v2["term3"] = 3.0 + +var query = new Vector +query["term2"] = 1.0 + var d1 = new Document("Doc 1", "/uri/1", v1) index.index_document(d1) @@ -77,27 +127,91 @@ var matches = index.match_vector(query) assert matches.first.document == d1 ~~~ -## StringIndex +## `StringIndex` -The StringIndex provides usefull services to index and match strings. +The [StringIndex](vsm::StringIndex) provides usefull services to index and match strings. ~~~ -index = new StringIndex +import vsm + +var index = new StringIndex -d1 = index.index_string("Doc 1", "/uri/1", "this is a sample") -d2 = index.index_string("Doc 2", "/uri/2", "this and this is another example") +var d1 = index.index_string("Doc 1", "/uri/1", "this is a sample") +var d2 = index.index_string("Doc 2", "/uri/2", "this and this is another example") assert index.documents.length == 2 -matches = index.match_string("this sample") +var matches = index.match_string("this sample") assert matches.first.document == d1 ~~~ -## FileIndex +Example from `vsm::example_vsm`: + +~~~ +# Example using a `FileIndex` +# +# This example shows of to index files from the system and retrieve them +# with text queries. +module example_vsm is example + +import vsm +import config + +redef class Config + + # --whitelist-exts + var opt_white_exts = new OptionArray("Allowed file extensions (default is [])", + "-w", "--whitelist-exts") + + # --blacklist-exts + var opt_black_exts = new OptionArray("Allowed file extensions (default is [])", + "-b", "--blacklist-exts") + + redef init do + opts.add_option(opt_white_exts, opt_black_exts) + end +end + +var config = new Config +config.tool_description = "usage: example_vsm " +config.parse_options(args) + +if args.length < 1 then + config.usage + exit 1 +end + +var index = new FileIndex +index.whitelist_exts = config.opt_white_exts.value +index.blacklist_exts = config.opt_black_exts.value + +print "Building index..." +index.index_files(args, true) +print "Indexed {index.documents.length} documents" + +loop + print "\nEnter query:" + printn "> " + var input = sys.stdin.read_line + var matches = index.match_string(input) + printn "" + for match in matches do + print match + end +end +~~~ -The FileIndex is a StringIndex able to index and retrieve files. +## `FileIndex` + +The [FileIndex](vsm::FileIndex) is a StringIndex able to index and retrieve files. ~~~nit -index = new FileIndex +import vsm + +var index = new FileIndex index.index_files(["/path/to/doc/1", "/path/to/doc/2"]) ~~~ + +## Authors + +This project is maintained by **Alexandre Terrasa **. diff --git a/uml-actors.svg b/uml-actors.svg new file mode 100644 index 0000000000..77076a546b --- /dev/null +++ b/uml-actors.svg @@ -0,0 +1,243 @@ + + + + + + +G + + +actors::actors_simple + + + +actors_simple ++ +A ++ +AMessagebar ++ +AMessagefoo ++ +ActorA ++ +MessageA ++ +ProxyA + + +actors::actors_mandelbrot + + + +actors_mandelbrot ++ +ActorWorker ++ +MessageWorker ++ +ProxyWorker ++ +Worker ++ +WorkerMessageget_byte ++ +WorkerMessageput_line ++ +WorkerMessagework + + +actors::actors_simple->actors::actors_mandelbrot + + + + +actors::actors_fannkuchredux + + + +actors_fannkuchredux ++ +ActorFannkuchRedux ++ +FannkuchRedux ++ +FannkuchReduxMessagecount ++ +FannkuchReduxMessagecount_flips ++ +FannkuchReduxMessagefirst_permutation ++ +FannkuchReduxMessagenext_permutation ++ +FannkuchReduxMessagep ++ +FannkuchReduxMessagepp ++ +FannkuchReduxMessageprint_p ++ +FannkuchReduxMessagerun ++ +FannkuchReduxMessagerun_task ++ +MessageFannkuchRedux ++ +ProxyFannkuchRedux + + +actors::actors_mandelbrot->actors::actors_fannkuchredux + + + + +actors::actors_thread_ring + + + +actors_thread_ring ++ +ActorThreadRing ++ +MessageThreadRing ++ +ProxyThreadRing ++ +ThreadRing ++ +ThreadRingMessageid ++ +ThreadRingMessagenext ++ +ThreadRingMessagesend_token + + +actors::actors_thread_ring->actors::actors_simple + + + + +actors::actors_simple_simulation + + + +actors_simple_simulation ++ +AgentMessagecount ++ +AgentMessagegreet ++ +AgentMessagegreet_back ++ +AgentMessageothers ++ +ProxyAgent + + +actors::actors_simple_simulation->actors::actors_thread_ring + + + + +actors::actors_agent_simulation + + + +actors_agent_simulation ++ +ActorAgent ++ +ActorClockAgent ++ +Agent ++ +AgentMessagedo_step ++ +AgentMessageend_step ++ +ClockAgent ++ +ClockAgentMessageagents ++ +ClockAgentMessagedo_step ++ +ClockAgentMessagefinished_step ++ +ClockAgentMessagenb_finished ++ +ClockAgentMessagenb_steps ++ +MessageAgent ++ +MessageClockAgent ++ +ProxyAgent ++ +ProxyClockAgent + + +actors::actors_chameneosredux + + + +actors_chameneosredux ++ +ActorCreature ++ +Creature ++ +CreatureMessagecolor ++ +CreatureMessagecount ++ +CreatureMessageid ++ +CreatureMessageplace ++ +CreatureMessagerun ++ +CreatureMessagesamecount ++ +CreatureMessageto_string ++ +MessageCreature ++ +ProxyCreature + + +actors::actors_chameneosredux->actors::actors_agent_simulation + + + + +actors::actors_fannkuchredux->actors::actors_chameneosredux + + + + +actors::actors + + + +actors ++ +Actor ++ +Future ++ +Mailbox ++ +Message ++ +Proxy ++ +ShutDownMessage ++ +SynchronizedCounter ++ +Sys + + + diff --git a/uml-ai.svg b/uml-ai.svg new file mode 100644 index 0000000000..302bdb98c7 --- /dev/null +++ b/uml-ai.svg @@ -0,0 +1,51 @@ + + + + + + +G + + +ai::ai + + + +ai + + +ai::backtrack + + + +backtrack ++ +BacktrackNode ++ +BacktrackProblem ++ +BacktrackSolver + + +ai::ai->ai::backtrack + + + + +ai::search + + + +search ++ +SearchNode ++ +SearchProblem ++ +SearchSolver + + + diff --git a/uml-android.svg b/uml-android.svg new file mode 100644 index 0000000000..0e4fef5fd5 --- /dev/null +++ b/uml-android.svg @@ -0,0 +1,934 @@ + + + + + + +G + + +android::portrait + + + +portrait + + +android::android + + + +android + + +android::portrait->android::android + + + + +android::landscape + + + +landscape + + +android::landscape->android::android + + + + +android::dalvik + + + +dalvik ++ +App ++ +Sys + + +android::android->android::dalvik + + + + +android::log + + + +log ++ +App ++ +Sys + + +android::android->android::log + + + + +android::at_boot + + + +at_boot + + +android::service + + + +service ++ +App ++ +NativeContext ++ +NativeService ++ +Service ++ +Sys + + +android::at_boot->android::service + + + + +android::shared_preferences + + + +shared_preferences + + +android::shared_preferences_api10 + + + +shared_preferences_api10 ++ +App ++ +NativeSharedPreferences ++ +NativeSharedPreferencesEditor ++ +SharedPreferences + + +android::shared_preferences->android::shared_preferences_api10 + + + + +android::intent_api12 + + + +intent_api12 ++ +Flag + + +android::intent_api11 + + + +intent_api11 ++ +Category ++ +Extra ++ +Flag + + +android::intent_api12->android::intent_api11 + + + + +android::intent_api17 + + + +intent_api17 ++ +Extra + + +android::intent_api16 + + + +intent_api16 ++ +Extra ++ +Flag ++ +Intent ++ +NativeIntent + + +android::intent_api17->android::intent_api16 + + + + +android::intent_api19 + + + +intent_api19 ++ +Extra ++ +Flag + + +android::intent_api18 + + + +intent_api18 ++ +Extra + + +android::intent_api19->android::intent_api18 + + + + +android::intent_api14 + + + +intent_api14 ++ +Extra + + +android::intent_api14->android::intent_api12 + + + + +android::platform + + + +platform ++ +Sys + + +android::aware + + + +aware + + +android::platform->android::aware + + + + +android::intent_api18->android::intent_api17 + + + + +android::activities + + + +activities ++ +NativeActivity ++ +NativeContext ++ +NativeContextWrapper + + +android::activities->android::platform + + + + +android::intent + + + +intent + + +android::intent_api11->android::intent + + + + +android::vibration + + + +vibration ++ +App ++ +Vibrator + + +android::vibration->android::dalvik + + + + +android::toast + + + +toast ++ +App + + +android::toast->android::dalvik + + + + +android::cardboard + + + +cardboard ++ +NativeHeadTracker + + +android::native_app_glue + + + +native_app_glue ++ +AConfiguration ++ +AInputQueue ++ +ALooper ++ +ANativeWindow ++ +App ++ +NativeAppGlue ++ +NativeNativeActivity ++ +NdkNativeActivity ++ +Sys + + +android::cardboard->android::native_app_glue + + + + +android::native_notification + + + +native_notification ++ +NativeContext ++ +NativeNotification ++ +NativeNotificationBuilder ++ +NativeNotificationManager + + +android::assets_and_resources + + + +assets_and_resources ++ +App ++ +AssetManager ++ +NativeAssetFileDescriptor ++ +NativeContext ++ +ResourcesManager + + +android::native_notification->android::assets_and_resources + + + + +android::data_store + + + +data_store ++ +DataStore + + +android::data_store->android::shared_preferences + + + + +android::intent_api15 + + + +intent_api15 ++ +Category ++ +Intent ++ +NativeIntent + + +android::intent_api15->android::intent_api14 + + + + +android::intent_api10 + + + +intent_api10 ++ +Action ++ +App ++ +Category ++ +Extra ++ +Flag ++ +Intent ++ +NativeComponentName ++ +NativeContext ++ +NativeIntent ++ +NativePackageManager ++ +Sys + + +android::intent->android::intent_api10 + + + + +android::key_event + + + +key_event ++ +NativeKeyEvent ++ +Sys + + +android::key_event->android::platform + + + + +android::wifi + + + +wifi ++ +NativeContext ++ +NativeListOfScanResult ++ +NativeScanResult ++ +NativeWifiManager + + +android::nit_activity + + + +nit_activity ++ +Activity ++ +App ++ +AppComponent ++ +NativeNitActivity ++ +Sys + + +android::wifi->android::nit_activity + + + + +android::game + + + +game ++ +App + + +android::assets + + + +assets ++ +NativeInputStream ++ +TextAsset + + +android::game->android::assets + + + + +android::game->android::native_app_glue + + + + +android::shared_preferences_api11 + + + +shared_preferences_api11 ++ +NativeSharedPreferences ++ +NativeSharedPreferencesEditor ++ +SharedPreferences + + +android::shared_preferences_api11->android::shared_preferences + + + + +android::dalvik->android::activities + + + + +android::assets->android::assets_and_resources + + + + +android::service->android::nit_activity + + + + +android::log->android::platform + + + + +android::intent_api16->android::intent_api15 + + + + +android::load_image + + + +load_image ++ +CByteArray ++ +Int ++ +NativeCByteArray + + +android::load_image->android::assets_and_resources + + + + +android::http_request + + + +http_request ++ +App ++ +AsyncHttpRequest ++ +JavaObject ++ +Sys ++ +Text + + +android::ui + + + +ui ++ +Activity ++ +Android_app_Fragment ++ +Android_widget_ArrayAdapter ++ +App ++ +Button ++ +CheckBox ++ +CompositeControl ++ +Control ++ +HorizontalLayout ++ +JavaString ++ +Label ++ +Layout ++ +ListLayout ++ +NativeActivity ++ +NativeButton ++ +NativeEditText ++ +Text ++ +TextInput ++ +TextView ++ +VerticalLayout ++ +View ++ +Window + + +android::http_request->android::ui + + + + +android::native_app_glue->android::dalvik + + + + +android::native_app_glue->android::log + + + + +android::nit_activity->android::key_event + + + + +android::nit_activity->android::dalvik + + + + +android::nit_activity->android::log + + + + +android::bundle + + + +bundle ++ +Array ++ +Bool ++ +Bundle ++ +Char ++ +Float ++ +Int ++ +NativeBundle ++ +Serializable ++ +String + + +android::nit_activity->android::bundle + + + + +android::notification + + + +notification ++ +Notification ++ +Sys + + +android::notification->android::native_notification + + + + +android::gamepad + + + +gamepad ++ +AndroidKeyEvent + + +android::input_events + + + +input_events ++ +AndroidInputEvent ++ +AndroidKeyEvent ++ +AndroidMotionEvent ++ +AndroidPointerEvent ++ +App + + +android::gamepad->android::input_events + + + + +android::ui->android::data_store + + + + +android::ui->android::assets + + + + +android::native_ui + + + +native_ui ++ +Android_app_Fragment ++ +Android_widget_AbsListView ++ +Android_widget_Adapter ++ +Android_widget_ArrayAdapter ++ +Android_widget_BaseAdapter ++ +Android_widget_CheckBox ++ +Android_widget_Checkable ++ +Android_widget_CompoundButton ++ +Android_widget_ListAdapter ++ +Android_widget_ListView ++ +Android_widget_SpinnerAdapter ++ +NativeActivity ++ +NativeButton ++ +NativeEditText ++ +NativeGridLayout ++ +NativeLinearLayout ++ +NativePopupWindow ++ +NativeTextView ++ +NativeView ++ +NativeViewGroup ++ +Sys + + +android::ui->android::native_ui + + + + +android::native_ui->android::nit_activity + + + + +android::input_events->android::game + + + + +android::sensors + + + +sensors ++ +ASensor ++ +ASensorAccelerometer ++ +ASensorEvent ++ +ASensorEventQueue ++ +ASensorEvents ++ +ASensorGyroscope ++ +ASensorLight ++ +ASensorMagneticField ++ +ASensorManager ++ +ASensorProximity ++ +ASensorType ++ +ASensorVector ++ +AndroidSensor ++ +App ++ +FullSensor + + +android::assets_and_resources->android::dalvik + + + + +android::shared_preferences_api10->android::dalvik + + + + +android::bundle->android::activities + + + + +android::audio + + + +audio ++ +App ++ +MediaPlayer ++ +Music ++ +PlayableAudio ++ +Sound ++ +SoundPool + + +android::audio->android::assets_and_resources + + + + +android::intent_api10->android::dalvik + + + + +android::intent_api10->android::bundle + + + + + diff --git a/uml-app.svg b/uml-app.svg new file mode 100644 index 0000000000..cc78960c19 --- /dev/null +++ b/uml-app.svg @@ -0,0 +1,168 @@ + + + + + + +G + + +app::app + + + +app + + +app::assets + + + +assets ++ +Asset ++ +TextAsset + + +app::app->app::assets + + + + +app::app_base + + + +app_base ++ +App ++ +AppComponent ++ +Sys + + +app::app->app::app_base + + + + +app::data_store + + + +data_store ++ +App ++ +DataStore + + +app::data_store->app::app_base + + + + +app::audio + + + +audio ++ +Music ++ +PlayableAudio ++ +Sound ++ +Sys + + +app::audio->app::app_base + + + + +app::http_request + + + +http_request ++ +App ++ +AsyncHttpRequest ++ +HttpRequestResult ++ +SimpleAsyncHttpRequest ++ +Text + + +app::http_request->app::app_base + + + + +app::ui + + + +ui ++ +App ++ +AppComponent ++ +AppEvent ++ +AppObserver ++ +Button ++ +ButtonPressEvent ++ +CheckBox ++ +CompositeControl ++ +Control ++ +HorizontalLayout ++ +Label ++ +Layout ++ +ListLayout ++ +Sys ++ +Text ++ +TextInput ++ +TextView ++ +ToggleEvent ++ +VerticalLayout ++ +View ++ +ViewEvent ++ +Window + + +app::ui->app::app_base + + + + + diff --git a/uml-core.svg b/uml-core.svg new file mode 100644 index 0000000000..72ee0d3f5a --- /dev/null +++ b/uml-core.svg @@ -0,0 +1,919 @@ + + + + + + +G + + +core::codecs + + + +codecs + + +core::iso8859_1 + + + +iso8859_1 ++ +Sys + + +core::codecs->core::iso8859_1 + + + + +core::utf8 + + + +utf8 ++ +Sys + + +core::codecs->core::utf8 + + + + +core::text + + + +text + + +core::fixed_ints_text + + + +fixed_ints_text ++ +Int16 ++ +Int32 ++ +Int8 ++ +Text ++ +UInt16 ++ +UInt32 + + +core::text->core::fixed_ints_text + + + + +core::ropes + + + +ropes ++ +FlatString ++ +Sys + + +core::text->core::ropes + + + + +core::bitset + + + +bitset ++ +Int + + +core::math + + + +math ++ +AbstractArray ++ +Byte ++ +Collection ++ +Comparable ++ +Float ++ +Int ++ +SequenceRead ++ +Sys + + +core::bitset->core::math + + + + +core::gc + + + +gc ++ +Finalizable ++ +FinalizableOnce ++ +Sys + + +core::kernel + + + +kernel ++ +Bool ++ +Byte ++ +Char ++ +Cloneable ++ +Comparable ++ +Discrete ++ +Float ++ +Int ++ +Numeric ++ +Object ++ +Pointer ++ +Sys ++ +Task + + +core::gc->core::kernel + + + + +core::codec_base + + + +codec_base ++ +Codec + + +core::codec_base->core::text + + + + +core::bytes + + + +bytes ++ +BytePattern ++ +Bytes ++ +CString ++ +FlatText ++ +Int ++ +Sys ++ +Text + + +core::codec_base->core::bytes + + + + +core::core + + + +core + + +core::core->core::bitset + + + + +core::environ + + + +environ ++ +CString ++ +String ++ +Sys + + +core::core->core::environ + + + + +core::protocol + + + +protocol ++ +DuplexProtocol ++ +Protocol ++ +ReaderProtocol ++ +WriterProtocol + + +core::core->core::protocol + + + + +core::numeric + + + +numeric ++ +Float ++ +Int ++ +Numeric ++ +Text + + +core::core->core::numeric + + + + +core::exec + + + +exec ++ +CString ++ +Process ++ +ProcessDuplex ++ +ProcessReader ++ +ProcessWriter ++ +Sys + + +core::core->core::exec + + + + +core::queue + + + +queue ++ +MinHeap ++ +Queue ++ +Sequence ++ +SimpleCollection + + +core::core->core::queue + + + + +core::re + + + +re ++ +CString ++ +Int ++ +Match ++ +Regex ++ +Sys ++ +Text + + +core::core->core::re + + + + +core::error + + + +error ++ +Error ++ +MaybeError + + +core::error->core::text + + + + +core::file + + + +file ++ +CString ++ +FileReader ++ +FileStat ++ +FileStream ++ +FileWriter ++ +FlatString ++ +Int ++ +Path ++ +Stderr ++ +Stdin ++ +Stdout ++ +Sys ++ +Text ++ +Writable + + +core::environ->core::file + + + + +core::stream + + + +stream ++ +Bytes ++ +BytesReader ++ +BytesWriter ++ +Duplex ++ +IOError ++ +LineIterator ++ +PollableReader ++ +Reader ++ +Stream ++ +StringReader ++ +StringWriter ++ +Text ++ +Writable ++ +Writer + + +core::protocol->core::stream + + + + +core::iso8859_1->core::codec_base + + + + +core::utf8->core::codec_base + + + + +core::collection + + + +collection ++ +Sequence + + +core::union_find + + + +union_find ++ +DisjointSet + + +core::collection->core::union_find + + + + +core::sorter + + + +sorter ++ +Comparator ++ +DefaultComparator ++ +DefaultReverseComparator ++ +MapComparator ++ +MapRead ++ +Sys + + +core::collection->core::sorter + + + + +core::circular_array + + + +circular_array ++ +CircularArray + + +core::collection->core::circular_array + + + + +core::list + + + +list ++ +List ++ +ListIterator + + +core::collection->core::list + + + + +core::numeric->core::text + + + + +core::range + + + +range ++ +Int ++ +Range + + +core::abstract_collection + + + +abstract_collection ++ +Collection ++ +Couple ++ +CoupleMap ++ +IndexedIterator ++ +Iterator ++ +IteratorDecorator ++ +Map ++ +MapIterator ++ +MapKeysIterator ++ +MapRead ++ +MapValuesIterator ++ +Ref ++ +RemovableCollection ++ +Sequence ++ +SequenceRead ++ +Set ++ +SimpleCollection + + +core::range->core::abstract_collection + + + + +core::time + + + +time ++ +Float ++ +ISODate ++ +Int ++ +Sys ++ +TimeT ++ +Tm + + +core::time->core::stream + + + + +core::exec->core::file + + + + +core::hash_collection + + + +hash_collection ++ +HashMap ++ +HashSet ++ +Map ++ +Set + + +core::union_find->core::hash_collection + + + + +core::math->core::collection + + + + +core::queue->core::math + + + + +core::string_search + + + +string_search ++ +BM_Pattern ++ +Char ++ +Match ++ +Pattern ++ +Text + + +core::fixed_ints_text->core::string_search + + + + +core::array + + + +array ++ +AbstractArray ++ +AbstractArrayRead ++ +Array ++ +ArrayCmp ++ +ArrayMap ++ +ArraySet ++ +Collection ++ +Iterator ++ +NativeArray + + +core::hash_collection->core::array + + + + +core::sorter->core::range + + + + +core::sorter->core::array + + + + +core::circular_array->core::array + + + + +core::native + + + +native ++ +CString ++ +Int ++ +UInt32 + + +core::native->core::math + + + + +core::fixed_ints + + + +fixed_ints ++ +Byte ++ +Float ++ +Int ++ +Int16 ++ +Int32 ++ +Int8 ++ +Numeric ++ +UInt16 ++ +UInt32 + + +core::native->core::fixed_ints + + + + +core::list->core::abstract_collection + + + + +core::re->core::gc + + + + +core::re->core::error + + + + +core::abstract_collection->core::kernel + + + + +core::abstract_text + + + +abstract_text ++ +Bool ++ +Buffer ++ +Byte ++ +CString ++ +CachedAlphaComparator ++ +Char ++ +Collection ++ +FlatText ++ +Float ++ +Int ++ +Map ++ +NativeArray ++ +Object ++ +String ++ +Sys ++ +Text + + +core::string_search->core::abstract_text + + + + +core::stream->core::codecs + + + + +core::stream->core::error + + + + +core::fixed_ints->core::kernel + + + + +core::array->core::abstract_collection + + + + +core::flat + + + +flat ++ +Array ++ +Buffer ++ +CString ++ +FlatBuffer ++ +FlatString ++ +FlatText ++ +Int ++ +Map ++ +NativeArray + + +core::ropes->core::flat + + + + +core::file->core::gc + + + + +core::file->core::time + + + + +core::bytes->core::flat + + + + +core::abstract_text->core::native + + + + +core::flat->core::abstract_text + + + + + diff --git a/uml-gamnit.svg b/uml-gamnit.svg new file mode 100644 index 0000000000..28789e4ed5 --- /dev/null +++ b/uml-gamnit.svg @@ -0,0 +1,1073 @@ + + + + + + +G + + +gamnit::landscape + + + +landscape + + +gamnit::portrait + + + +portrait + + +gamnit::common + + + +common ++ +Sys + + +gamnit::camera_control + + + +camera_control ++ +EulerCamera + + +gamnit::gamnit + + + +gamnit ++ +App ++ +Sys + + +gamnit::camera_control->gamnit::gamnit + + + + +gamnit::cameras + + + +cameras ++ +Camera ++ +EulerCamera ++ +UICamera + + +gamnit::camera_control->gamnit::cameras + + + + +gamnit::network + + + +network + + +gamnit::client + + + +client ++ +RemoteServer ++ +RemoteServerConfig ++ +Sys + + +gamnit::network->gamnit::client + + + + +gamnit::server + + + +server ++ +RemoteClient ++ +Server + + +gamnit::network->gamnit::server + + + + +gamnit::vr + + + +vr + + +gamnit::cardboard + + + +cardboard ++ +App ++ +EulerCamera + + +gamnit::vr->gamnit::cardboard + + + + +gamnit::stereoscopic_view + + + +stereoscopic_view ++ +App ++ +EulerCamera ++ +GamnitDisplay + + +gamnit::vr->gamnit::stereoscopic_view + + + + +gamnit::android19 + + + +android19 ++ +NativeCByteArray ++ +TextureAsset + + +gamnit::gamnit_android + + + +gamnit_android ++ +App ++ +GamnitDisplay ++ +NativeActivity ++ +Sys + + +gamnit::android19->gamnit::gamnit_android + + + + +gamnit::gamnit_ios + + + +gamnit_ios ++ +App ++ +GamnitGLKView ++ +Sys + + +gamnit::display_ios + + + +display_ios ++ +GamnitDisplay ++ +GamnitGLKView ++ +TextureAsset + + +gamnit::gamnit_ios->gamnit::display_ios + + + + +gamnit::gamnit_ios->gamnit::gamnit + + + + +gamnit::display + + + +display ++ +GamnitDisplay ++ +Sys + + +gamnit::flat + + + +flat + + +gamnit::flat->gamnit::camera_control + + + + +gamnit::keys + + + +keys ++ +App + + +gamnit::flat->gamnit::keys + + + + +gamnit::limit_fps + + + +limit_fps ++ +App + + +gamnit::flat->gamnit::limit_fps + + + + +gamnit::tileset + + + +tileset ++ +TileSet ++ +TileSetFont + + +gamnit::flat->gamnit::tileset + + + + +gamnit::bmfont + + + +bmfont ++ +BMFont ++ +BMFontAsset ++ +BMFontChar ++ +Text + + +gamnit::flat->gamnit::bmfont + + + + +gamnit::keys->gamnit::gamnit + + + + +gamnit::depth_core + + + +depth_core ++ +Actor ++ +App ++ +CompositeModel ++ +LeafModel ++ +Light ++ +LightCastingShadows ++ +Material ++ +Mesh ++ +Model ++ +PointLight + + +gamnit::flat_core + + + +flat_core ++ +Animation ++ +App ++ +GLfloatArray ++ +NativeGLfloatArray ++ +Point3d ++ +Sprite ++ +SpriteSet ++ +Sys ++ +Texture + + +gamnit::depth_core->gamnit::flat_core + + + + +gamnit::limit_fps->gamnit::gamnit + + + + +gamnit::camera_control_linux + + + +camera_control_linux ++ +EulerCamera + + +gamnit::camera_control_linux->gamnit::camera_control + + + + +gamnit::gamnit_linux + + + +gamnit_linux ++ +App ++ +GamnitDisplay ++ +GamnitInputEvent ++ +GamnitKeyEvent ++ +GamnitMouseWheelEvent ++ +GamnitOtherEvent ++ +GamnitPointerEvent ++ +GamnitQuitEvent ++ +SDLEvent ++ +SDLKeyboardEvent ++ +SDLMouseEvent ++ +SDLMouseWheelEvent ++ +SDLQuitEvent + + +gamnit::camera_control_linux->gamnit::gamnit_linux + + + + +gamnit::input_ios + + + +input_ios ++ +GamnitIOSPointerEvent ++ +NitGLKView + + +gamnit::input_ios->gamnit::gamnit_ios + + + + +gamnit::textures + + + +textures ++ +AbsoluteSubtexture ++ +CheckerTexture ++ +CustomTexture ++ +Pointer ++ +RelativeSubtexture ++ +RootTexture ++ +Subtexture ++ +Sys ++ +Texture ++ +TextureAsset ++ +TextureSet + + +gamnit::display_ios->gamnit::textures + + + + +gamnit::gamnit->gamnit::textures + + + + +gamnit::programs + + + +programs ++ +Attribute ++ +AttributeFloat ++ +AttributeMap ++ +AttributeVec2 ++ +AttributeVec3 ++ +AttributeVec4 ++ +FragmentShader ++ +GamnitProgram ++ +GamnitProgramFromSource ++ +InactiveAttribute ++ +InactiveUniform ++ +InactiveVariable ++ +Matrix ++ +NativeGLfloatArray ++ +Shader ++ +ShaderVariable ++ +ShaderVariableMap ++ +Sys ++ +Uniform ++ +UniformBool ++ +UniformFloat ++ +UniformInt ++ +UniformMap ++ +UniformMat4 ++ +UniformSampler2D ++ +UniformVec2 ++ +UniformVec3 ++ +UniformVec4 ++ +VertexShader + + +gamnit::gamnit->gamnit::programs + + + + +gamnit::display_android + + + +display_android ++ +GamnitDisplay ++ +Pointer ++ +TextureAsset + + +gamnit::display_android->gamnit::textures + + + + +gamnit::egl + + + +egl ++ +GamnitDisplay + + +gamnit::display_android->gamnit::egl + + + + +gamnit::font + + + +font ++ +Font ++ +Sys ++ +TextSprites + + +gamnit::font->gamnit::flat_core + + + + +gamnit::depth + + + +depth ++ +App + + +gamnit::cardboard->gamnit::depth + + + + +gamnit::stereoscopic_view->gamnit::depth + + + + +gamnit::texture_atlas_parser + + + +texture_atlas_parser ++ +Sys + + +gamnit::client->gamnit::common + + + + +gamnit::cameras_cache + + + +cameras_cache ++ +Camera ++ +EulerCamera ++ +UICamera + + +gamnit::cameras_cache->gamnit::cameras + + + + +gamnit::particles + + + +particles ++ +App ++ +ExplosionProgram ++ +ParticleProgram ++ +ParticleSystem ++ +SmokeProgram + + +gamnit::depth->gamnit::particles + + + + +gamnit::selection + + + +selection ++ +App ++ +Material ++ +SelectionProgram ++ +TexturedMaterial + + +gamnit::depth->gamnit::selection + + + + +gamnit::more_models + + + +more_models ++ +Model ++ +ModelAsset ++ +Sys + + +gamnit::depth->gamnit::more_models + + + + +gamnit::mtl + + + +mtl ++ +MtlDef ++ +MtlFileParser + + +gamnit::model_parser_base + + + +model_parser_base ++ +StringProcessor ++ +Vec3 ++ +Vec4 + + +gamnit::mtl->gamnit::model_parser_base + + + + +gamnit::gamnit_linux->gamnit::gamnit + + + + +gamnit::display_linux + + + +display_linux ++ +GamnitDisplay ++ +TextureAsset + + +gamnit::gamnit_linux->gamnit::display_linux + + + + +gamnit::camera_control_android + + + +camera_control_android ++ +EulerCamera ++ +Point + + +gamnit::camera_control_android->gamnit::camera_control + + + + +gamnit::camera_control_android->gamnit::gamnit_android + + + + +gamnit::display_linux->gamnit::textures + + + + +gamnit::display_linux->gamnit::egl + + + + +gamnit::server->gamnit::common + + + + +gamnit::particles->gamnit::depth_core + + + + +gamnit::tileset->gamnit::font + + + + +gamnit::virtual_gamepad_spritesheet + + + +virtual_gamepad_spritesheet ++ +VirtualGamepadSpritesheet + + +gamnit::virtual_gamepad_spritesheet->gamnit::textures + + + + +gamnit::obj + + + +obj ++ +ObjDef ++ +ObjFace ++ +ObjFileParser ++ +ObjObj ++ +ObjVertex + + +gamnit::obj->gamnit::model_parser_base + + + + +gamnit::more_lights + + + +more_lights ++ +ParallelLight + + +gamnit::more_lights->gamnit::depth_core + + + + +gamnit::gamnit_android->gamnit::gamnit + + + + +gamnit::gamnit_android->gamnit::display_android + + + + +gamnit::model_dimensions + + + +model_dimensions ++ +LeafModel ++ +Mesh ++ +Model + + +gamnit::model_dimensions->gamnit::depth_core + + + + +gamnit::textures->gamnit::display + + + + +gamnit::egl->gamnit::display + + + + +gamnit::more_materials + + + +more_materials ++ +App ++ +BlinnPhongProgram ++ +Material ++ +NormalProgram ++ +NormalsMaterial ++ +SmoothMaterial ++ +TexturedMaterial + + +gamnit::selection->gamnit::more_materials + + + + +gamnit::more_meshes + + + +more_meshes ++ +Boxed3d ++ +Cube ++ +Cuboid ++ +Plane ++ +UVSphere + + +gamnit::more_meshes->gamnit::model_dimensions + + + + +gamnit::dynamic_resolution + + + +dynamic_resolution ++ +App + + +gamnit::dynamic_resolution->gamnit::gamnit + + + + +gamnit::virtual_gamepad + + + +virtual_gamepad ++ +App ++ +DPad ++ +RoundButton ++ +RoundControl ++ +VirtualGamepad ++ +VirtualGamepadEvent + + +gamnit::virtual_gamepad->gamnit::flat + + + + +gamnit::virtual_gamepad->gamnit::virtual_gamepad_spritesheet + + + + +gamnit::programs->gamnit::display + + + + +gamnit::cameras->gamnit::display + + + + +gamnit::bmfont->gamnit::font + + + + +gamnit::shadow + + + +shadow ++ +App ++ +Material ++ +ShadowDepthProgram + + +gamnit::shadow->gamnit::depth_core + + + + +gamnit::more_models->gamnit::mtl + + + + +gamnit::more_models->gamnit::obj + + + + +gamnit::more_models->gamnit::more_meshes + + + + +gamnit::more_models->gamnit::more_materials + + + + +gamnit::more_materials->gamnit::flat + + + + +gamnit::more_materials->gamnit::more_lights + + + + +gamnit::more_materials->gamnit::shadow + + + + +gamnit::flat_core->gamnit::cameras_cache + + + + +gamnit::flat_core->gamnit::dynamic_resolution + + + + + diff --git a/uml-geometry.svg b/uml-geometry.svg new file mode 100644 index 0000000000..632431cf5e --- /dev/null +++ b/uml-geometry.svg @@ -0,0 +1,141 @@ + + + + + + +G + + +geometry::geometry + + + +geometry + + +geometry::angles + + + +angles ++ +Float ++ +Point + + +geometry::geometry->geometry::angles + + + + +geometry::boxes + + + +boxes ++ +Box ++ +Box3d ++ +Boxed ++ +Boxed3d ++ +BoxedArray ++ +BoxedCollection ++ +ILine ++ +ILine3d ++ +IPoint ++ +IPoint3d + + +geometry::geometry->geometry::boxes + + + + +geometry::points_and_lines + + + +points_and_lines ++ +Deserializer ++ +ILine ++ +ILine3d ++ +IPoint ++ +IPoint3d ++ +Line ++ +Line3d ++ +Point ++ +Point3d + + +geometry::angles->geometry::points_and_lines + + + + +geometry::boxes->geometry::points_and_lines + + + + +geometry::quadtree + + + +quadtree ++ +DQuadTree ++ +QuadTree ++ +SQuadTree + + +geometry::quadtree->geometry::boxes + + + + +geometry::polygon + + + +polygon ++ +APolygon ++ +ConvexPolygon ++ +Polygon ++ +Sys + + +geometry::polygon->geometry::points_and_lines + + + + + diff --git a/uml-github.svg b/uml-github.svg new file mode 100644 index 0000000000..3b5d4e054c --- /dev/null +++ b/uml-github.svg @@ -0,0 +1,243 @@ + + + + + + +G + + +github::github + + + +github + + +github::cache + + + +cache ++ +GithubAPI + + +github::github->github::cache + + + + +github::api + + + +api ++ +Branch ++ +Comment ++ +Commit ++ +CommitComment ++ +ContributorStats ++ +Deserializer ++ +GitCommit ++ +GitUser ++ +GithubAPI ++ +GithubDeserializer ++ +GithubEntity ++ +GithubFile ++ +ISODate ++ +Issue ++ +IssueComment ++ +IssueEvent ++ +Label ++ +Milestone ++ +PullRef ++ +PullRequest ++ +RenameAction ++ +Repo ++ +ReviewComment ++ +User + + +github::cache->github::api + + + + +github::github_curl + + + +github_curl ++ +GithubCurl ++ +GithubError ++ +Sys + + +github::wallet + + + +wallet ++ +GithubWallet + + +github::wallet->github::github + + + + +github::hooks + + + +hooks ++ +HookListener + + +github::events + + + +events ++ +CommitCommentEvent ++ +CreateEvent ++ +DeleteEvent ++ +DeploymentEvent ++ +DeploymentStatusEvent ++ +Deserializer ++ +ForkEvent ++ +GithubDeserializer ++ +GithubEvent ++ +IssueCommentEvent ++ +IssuesEvent ++ +MemberEvent ++ +PullRequestEvent ++ +PullRequestReviewCommentEvent ++ +PushEvent ++ +StatusEvent + + +github::hooks->github::events + + + + +github::events->github::api + + + + +github::loader + + + +loader ++ +Branch ++ +BranchRepo ++ +Commit ++ +CommitRepo ++ +Deserializer ++ +GithubWallet ++ +Issue ++ +IssueComment ++ +IssueCommentRepo ++ +IssueEvent ++ +IssueEventRepo ++ +IssueRepo ++ +Loader ++ +LoaderConfig ++ +LoaderJob ++ +LoaderJobRepo ++ +PullRequestRepo ++ +Repo ++ +RepoEntity ++ +RepoRepo ++ +Sys + + +github::loader->github::wallet + + + + +github::loader->github::events + + + + +github::api->github::github_curl + + + + + diff --git a/uml-ios.svg b/uml-ios.svg new file mode 100644 index 0000000000..5351e736e8 --- /dev/null +++ b/uml-ios.svg @@ -0,0 +1,242 @@ + + + + + + +G + + +ios::platform + + + +platform + + +ios::ios + + + +ios ++ +NSString ++ +Sys ++ +Text + + +ios::app + + + +app ++ +App ++ +AppDelegate ++ +Sys ++ +UIApplication + + +ios::ios->ios::app + + + + +ios::http_request + + + +http_request ++ +App ++ +NSString ++ +Text + + +ios::http_request->ios::ios + + + + +ios::assets + + + +assets ++ +Sys ++ +TextAsset + + +ios::data_store + + + +data_store ++ +DataStore + + +ios::app->ios::platform + + + + +ios::audio + + + +audio ++ +PlayableAudio + + +ios::audio->ios::assets + + + + +ios::glkit + + + +glkit ++ +NSSet_UITouch ++ +NitGLKView ++ +UIEvent ++ +UITouch + + +ios::glkit->ios::ios + + + + +ios::uikit + + + +uikit ++ +UIButton ++ +UIButtonType ++ +UIColor ++ +UIControl ++ +UILabel ++ +UILayoutConstraintAxis ++ +UIScrollView ++ +UIStackView ++ +UIStackViewAlignment ++ +UIStackViewDistribution ++ +UISwitch ++ +UITableView ++ +UITableViewCell ++ +UITableViewDataSource ++ +UITableViewDelegate ++ +UITableViewStyle ++ +UITextField ++ +UIView ++ +UIViewController ++ +UIWindow + + +ios::uikit->ios::ios + + + + +ios::ui + + + +ui ++ +App ++ +AppDelegate ++ +Button ++ +CheckBox ++ +CompositeControl ++ +Control ++ +HorizontalLayout ++ +Label ++ +Layout ++ +ListLayout ++ +NSString ++ +NitViewController ++ +TableView ++ +Text ++ +TextInput ++ +TextView ++ +UIButton ++ +UILabel ++ +UISwitch ++ +UITableView ++ +UITextField ++ +VerticalLayout ++ +View ++ +Window + + +ios::ui->ios::uikit + + + + + diff --git a/uml-json.svg b/uml-json.svg new file mode 100644 index 0000000000..e4c76d6b13 --- /dev/null +++ b/uml-json.svg @@ -0,0 +1,154 @@ + + + + + + +G + + +json::json + + + +json + + +json::serialization_read + + + +serialization_read ++ +JsonDeserializer ++ +Map ++ +SimpleCollection ++ +Sys ++ +Text + + +json::json->json::serialization_read + + + + +json::serialization_write + + + +serialization_write ++ +Bool ++ +Byte ++ +CString ++ +Char ++ +Collection ++ +Float ++ +Int ++ +JsonSerializer ++ +Map ++ +Serializable ++ +SimpleCollection ++ +Text + + +json::json->json::serialization_write + + + + +json::error + + + +error ++ +Deserializer ++ +JsonParseError + + +json::store + + + +store ++ +JsonStore + + +json::store->json::json + + + + +json::dynamic + + + +dynamic ++ +JsonKeyError ++ +JsonValue ++ +Text + + +json::static + + + +static ++ +Char ++ +FlatText ++ +JSONStringParser ++ +JsonArray ++ +JsonMapRead ++ +JsonObject ++ +JsonSequenceRead ++ +Text + + +json::dynamic->json::static + + + + +json::serialization_read->json::static + + + + +json::static->json::error + + + + + diff --git a/uml-markdown.svg b/uml-markdown.svg new file mode 100644 index 0000000000..05c81dc0ab --- /dev/null +++ b/uml-markdown.svg @@ -0,0 +1,197 @@ + + + + + + +G + + +markdown::nitmd + + + +nitmd ++ +Sys + + +markdown::man + + + +man ++ +ManDecorator + + +markdown::nitmd->markdown::man + + + + +markdown::decorators + + + +decorators ++ +InlineDecorator ++ +MdDecorator + + +markdown::nitmd->markdown::decorators + + + + +markdown::wikilinks + + + +wikilinks ++ +Decorator ++ +MarkdownProcessor ++ +TokenWikiLink + + +markdown::markdown + + + +markdown ++ +Block ++ +BlockCode ++ +BlockFence ++ +BlockHeadline ++ +BlockList ++ +BlockListItem ++ +BlockNone ++ +BlockOrderedList ++ +BlockParagraph ++ +BlockQuote ++ +BlockRuler ++ +BlockUnorderedList ++ +BlockXML ++ +Decorator ++ +HTMLDecorator ++ +HeadLine ++ +Line ++ +LineBlockquote ++ +LineCode ++ +LineEmpty ++ +LineFence ++ +LineHR ++ +LineHeadline ++ +LineHeadline1 ++ +LineHeadline2 ++ +LineList ++ +LineOList ++ +LineOther ++ +LineUList ++ +LineXML ++ +LinkRef ++ +MDBlock ++ +MDLine ++ +MDLocation ++ +MarkdownProcessor ++ +String ++ +Text ++ +Token ++ +TokenCode ++ +TokenCodeDouble ++ +TokenCodeSingle ++ +TokenEm ++ +TokenEmStar ++ +TokenEmUnderscore ++ +TokenEntity ++ +TokenEscape ++ +TokenHTML ++ +TokenImage ++ +TokenLink ++ +TokenLinkOrImage ++ +TokenNone ++ +TokenStrike ++ +TokenStrong ++ +TokenStrongStar ++ +TokenStrongUnderscore + + +markdown::wikilinks->markdown::markdown + + + + +markdown::man->markdown::markdown + + + + +markdown::decorators->markdown::markdown + + + + + diff --git a/uml-nitcorn.svg b/uml-nitcorn.svg new file mode 100644 index 0000000000..b4f271adc7 --- /dev/null +++ b/uml-nitcorn.svg @@ -0,0 +1,329 @@ + + + + + + +G + + +nitcorn::pthreads + + + +pthreads ++ +HttpFactory + + +nitcorn::nitcorn + + + +nitcorn + + +nitcorn::pthreads->nitcorn::nitcorn + + + + +nitcorn::signal_handler + + + +signal_handler ++ +HttpFactory + + +nitcorn::nitcorn->nitcorn::signal_handler + + + + +nitcorn::file_server + + + +file_server ++ +FileServer ++ +String + + +nitcorn::nitcorn->nitcorn::file_server + + + + +nitcorn::reactor + + + +reactor ++ +Action ++ +HttpFactory ++ +HttpServer ++ +Interface ++ +Interfaces ++ +ServerConfig ++ +Sys ++ +VirtualHosts + + +nitcorn::signal_handler->nitcorn::reactor + + + + +nitcorn::token + + + +token ++ +Sys + + +nitcorn::proxy + + + +proxy ++ +ProxyAction + + +nitcorn::proxy->nitcorn::reactor + + + + +nitcorn::server_config + + + +server_config ++ +Action ++ +Interface ++ +Interfaces ++ +Route ++ +Routes ++ +ServerConfig ++ +VirtualHost ++ +VirtualHosts + + +nitcorn::restful + + + +restful ++ +RestfulAction ++ +RestfulTask + + +nitcorn::restful->nitcorn::nitcorn + + + + +nitcorn::sessions + + + +sessions ++ +HttpRequest ++ +HttpRequestParser ++ +HttpResponse ++ +Session ++ +Sys + + +nitcorn::sessions->nitcorn::token + + + + +nitcorn::sessions->nitcorn::server_config + + + + +nitcorn::http_request + + + +http_request ++ +Deserializer ++ +HttpRequest ++ +HttpRequestParser + + +nitcorn::sessions->nitcorn::http_request + + + + +nitcorn::http_response + + + +http_response ++ +Deserializer ++ +HttpResponse ++ +HttpStatusCodes ++ +Sys + + +nitcorn::sessions->nitcorn::http_response + + + + +nitcorn::http_errors + + + +http_errors ++ +ErrorTemplate + + +nitcorn::http_errors->nitcorn::http_response + + + + +nitcorn::log + + + +log ++ +Action ++ +HttpServer ++ +Sys + + +nitcorn::log->nitcorn::reactor + + + + +nitcorn::http_request_buffer + + + +http_request_buffer ++ +HTTPConnection + + +nitcorn::reactor->nitcorn::http_request_buffer + + + + +nitcorn::vararg_routes + + + +vararg_routes ++ +HttpRequest ++ +Route ++ +Routes + + +nitcorn::reactor->nitcorn::vararg_routes + + + + +nitcorn::reactor->nitcorn::http_response + + + + +nitcorn::vararg_routes->nitcorn::server_config + + + + +nitcorn::vararg_routes->nitcorn::http_request + + + + +nitcorn::file_server->nitcorn::sessions + + + + +nitcorn::file_server->nitcorn::http_errors + + + + +nitcorn::file_server->nitcorn::reactor + + + + +nitcorn::media_types + + + +media_types ++ +MediaTypes ++ +Sys + + +nitcorn::file_server->nitcorn::media_types + + + + + diff --git a/uml-nlp.svg b/uml-nlp.svg new file mode 100644 index 0000000000..31357761b9 --- /dev/null +++ b/uml-nlp.svg @@ -0,0 +1,82 @@ + + + + + + +G + + +nlp::nlp_server + + + +nlp_server ++ +Config ++ +Sys + + +nlp::nlp + + + +nlp ++ +NLPFileIndex ++ +NLPIndex + + +nlp::nlp_server->nlp::nlp + + + + +nlp::stanford + + + +stanford ++ +NLPClient ++ +NLPDocument ++ +NLPJavaProcessor ++ +NLPProcessor ++ +NLPSentence ++ +NLPServer ++ +NLPToken + + +nlp::nlp->nlp::stanford + + + + +nlp::nlp_index + + + +nlp_index ++ +Config ++ +Sys + + +nlp::nlp_index->nlp::nlp + + + + + diff --git a/uml-popcorn.svg b/uml-popcorn.svg new file mode 100644 index 0000000000..9b89a0c217 --- /dev/null +++ b/uml-popcorn.svg @@ -0,0 +1,332 @@ + + + + + + +G + + +popcorn::pop_sessions + + + +pop_sessions ++ +SessionInit + + +popcorn::pop_handlers + + + +pop_handlers ++ +App ++ +Handler ++ +HttpResponse ++ +Router ++ +StaticHandler + + +popcorn::pop_sessions->popcorn::pop_handlers + + + + +popcorn::pop_tasks + + + +pop_tasks ++ +App ++ +PopTask + + +popcorn::pop_tasks->popcorn::pop_handlers + + + + +popcorn::pop_json + + + +pop_json ++ +Handler ++ +HttpResponse + + +popcorn::pop_json->popcorn::pop_handlers + + + + +popcorn::pop_validation + + + +pop_validation ++ +ArrayField ++ +ArrayValidator ++ +BoolField ++ +DocumentValidator ++ +EmailField ++ +FieldValidator ++ +FieldsMatch ++ +FloatField ++ +ISBNField ++ +IntField ++ +ObjectField ++ +ObjectValidator ++ +RegexField ++ +RequiredField ++ +StringField ++ +URLField ++ +UniqueField ++ +ValidationResult + + +popcorn::pop_json->popcorn::pop_validation + + + + +popcorn::pop_templates + + + +pop_templates ++ +HttpResponse + + +popcorn::pop_templates->popcorn::pop_json + + + + +popcorn::pop_config + + + +pop_config ++ +AppConfig + + +popcorn::popcorn + + + +popcorn ++ +App ++ +ErrorTpl + + +popcorn::popcorn->popcorn::pop_sessions + + + + +popcorn::popcorn->popcorn::pop_tasks + + + + +popcorn::pop_logging + + + +pop_logging ++ +ConsoleLog ++ +HttpRequest ++ +HttpResponse ++ +RequestClock + + +popcorn::popcorn->popcorn::pop_logging + + + + +popcorn::pop_tests + + + +pop_tests ++ +AppThread ++ +ClientThread ++ +Sys ++ +TestPopcorn + + +popcorn::pop_tests->popcorn::popcorn + + + + +popcorn::pop_routes + + + +pop_routes ++ +AppGlobRoute ++ +AppParamRoute ++ +AppRoute + + +popcorn::pop_repos + + + +pop_repos ++ +AppConfig ++ +Deserializer ++ +JsonObject ++ +JsonRepository ++ +MongoRepository ++ +RepoObject ++ +Repository ++ +RepositoryQuery + + +popcorn::pop_repos->popcorn::pop_config + + + + +popcorn::pop_auth + + + +pop_auth ++ +AuthHandler ++ +GithubLogin ++ +GithubLogout ++ +GithubOAuthCallBack ++ +GithubUser ++ +Session + + +popcorn::pop_auth->popcorn::pop_sessions + + + + +popcorn::pop_auth->popcorn::pop_json + + + + +popcorn::pop_logging->popcorn::pop_handlers + + + + +popcorn::pop_tracker + + + +pop_tracker ++ +AppConfig ++ +Deserializer ++ +LogEntry ++ +PopTracker ++ +PopTrackerAPI ++ +PopTrackerBrowsers ++ +PopTrackerEntries ++ +PopTrackerQueries ++ +PopTrackerResponseTime ++ +TrackerHandler ++ +TrackerRepo + + +popcorn::pop_tracker->popcorn::pop_json + + + + +popcorn::pop_tracker->popcorn::popcorn + + + + +popcorn::pop_tracker->popcorn::pop_repos + + + + +popcorn::pop_handlers->popcorn::pop_routes + + + + + diff --git a/uml-posix.svg b/uml-posix.svg new file mode 100644 index 0000000000..6c66b45a8a --- /dev/null +++ b/uml-posix.svg @@ -0,0 +1,40 @@ + + + + + + +G + + +posix::ext + + + +ext ++ +Passwd + + +posix::posix + + + +posix ++ +Group ++ +Passwd ++ +Sys + + +posix::ext->posix::posix + + + + + diff --git a/uml-pthreads.svg b/uml-pthreads.svg new file mode 100644 index 0000000000..89c392b958 --- /dev/null +++ b/uml-pthreads.svg @@ -0,0 +1,118 @@ + + + + + + +G + + +pthreads::extra + + + +extra ++ +Sys ++ +Thread + + +pthreads::pthreads + + + +pthreads ++ +AtomicInt ++ +Barrier ++ +MainThread ++ +Mutex ++ +PthreadCond ++ +Sys ++ +Thread + + +pthreads::extra->pthreads::pthreads + + + + +pthreads::threadpool + + + +threadpool ++ +JoinTask ++ +Task ++ +ThreadPool + + +pthreads::concurrent_collections + + + +concurrent_collections ++ +Array ++ +BlockingQueue ++ +Collection ++ +ConcurrentArray ++ +ConcurrentCollection ++ +ConcurrentList ++ +ConcurrentSequence ++ +ConcurrentSequenceRead ++ +List ++ +ReverseBlockingQueue ++ +Sequence ++ +SequenceRead + + +pthreads::threadpool->pthreads::concurrent_collections + + + + +pthreads::redef_collections + + + +redef_collections ++ +Array + + +pthreads::redef_collections->pthreads::pthreads + + + + +pthreads::concurrent_collections->pthreads::pthreads + + + + + diff --git a/uml-sdl2.svg b/uml-sdl2.svg new file mode 100644 index 0000000000..2574fddb7e --- /dev/null +++ b/uml-sdl2.svg @@ -0,0 +1,181 @@ + + + + + + +G + + +sdl2::all + + + +all + + +sdl2::image + + + +image ++ +IMG ++ +SDL ++ +SDLImgInitFlags ++ +SDLSurface + + +sdl2::all->sdl2::image + + + + +sdl2::sdl2 + + + +sdl2 + + +sdl2::syswm + + + +syswm ++ +SDLSysWMInfo ++ +SDLWindow + + +sdl2::sdl2->sdl2::syswm + + + + +sdl2::events + + + +events ++ +SDLEvent ++ +SDLEventBuffer ++ +SDLKeyboardDownEvent ++ +SDLKeyboardEvent ++ +SDLKeyboardUpEvent ++ +SDLKeysym ++ +SDLMouseButtonDownEvent ++ +SDLMouseButtonEvent ++ +SDLMouseButtonUpEvent ++ +SDLMouseEvent ++ +SDLMouseMotionEvent ++ +SDLMouseWheelEvent ++ +SDLQuitEvent ++ +SDLWindowEvent + + +sdl2::sdl2->sdl2::events + + + + +sdl2::sdl2_base + + + +sdl2_base ++ +Int ++ +SDL ++ +SDLColor ++ +SDLInitFlags ++ +SDLMessageBoxFlags ++ +SDLPoint ++ +SDLRect ++ +SDLRenderer ++ +SDLRendererFlags ++ +SDLRendererInfo ++ +SDLSurface ++ +SDLTexture ++ +SDLWindow ++ +SDLWindowFlags ++ +SDL_PixelFormat ++ +Sys + + +sdl2::syswm->sdl2::sdl2_base + + + + +sdl2::image->sdl2::sdl2 + + + + +sdl2::mixer + + + +mixer ++ +Mix ++ +MixChunk ++ +MixFormat ++ +MixInitFlags ++ +MixMusic ++ +Sys + + +sdl2::mixer->sdl2::sdl2_base + + + + +sdl2::events->sdl2::sdl2_base + + + + + diff --git a/uml-serialization.svg b/uml-serialization.svg new file mode 100644 index 0000000000..9912144945 --- /dev/null +++ b/uml-serialization.svg @@ -0,0 +1,170 @@ + + + + + + +G + + +serialization::serialization + + + +serialization + + +serialization::inspect + + + +inspect ++ +Bool ++ +Byte ++ +CString ++ +Char ++ +Collection ++ +Float ++ +Int ++ +Map ++ +Serializable ++ +SimpleCollection ++ +Sys ++ +Text + + +serialization::serialization->serialization::inspect + + + + +serialization::engine_tools + + + +engine_tools ++ +Object ++ +StrictHashMap ++ +String ++ +Text + + +serialization::serialization_core + + + +serialization_core ++ +AttributeError ++ +AttributeMissingError ++ +AttributeTypeError ++ +Bool ++ +Byte ++ +CString ++ +Char ++ +Couple ++ +Deserializer ++ +DirectSerializable ++ +Error ++ +Float ++ +Int ++ +Map ++ +MinHeap ++ +Ref ++ +Serializable ++ +Serializer ++ +SimpleCollection ++ +Text + + +serialization::engine_tools->serialization::serialization_core + + + + +serialization::safe + + + +safe ++ +Deserializer ++ +SafeDeserializer ++ +Sys + + +serialization::safe->serialization::serialization + + + + +serialization::caching + + + +caching ++ +AsyncCache ++ +CachingDeserializer ++ +CachingSerializer ++ +DeserializerCache ++ +DuplexCache ++ +SerializerCache + + +serialization::caching->serialization::engine_tools + + + + +serialization::inspect->serialization::caching + + + + + diff --git a/uml-vsm.svg b/uml-vsm.svg new file mode 100644 index 0000000000..ace1c99945 --- /dev/null +++ b/uml-vsm.svg @@ -0,0 +1,104 @@ + + + + + + +G + + +vsm::IndexMatchSorter + + +IndexMatchSorter + + +vsm::IndexMatch + + +IndexMatch[DOC: Document] + + +vsm::StringIndex + + +StringIndex + ++ + index_string(title: String, uri: String, string: String, auto_update: nullable Bool): DOC ++ + match_string(query: String): Array[IndexMatch[DOC]] ++ + parse_string(string: String): Vector + + +vsm::VSMIndex + + +VSMIndex + ++ + DOC: Document + ++ + index_document(doc: DOC, auto_update: nullable Bool) ++ + match_vector(query: Vector): Array[IndexMatch[DOC]] ++ + update_index() + + +vsm::StringIndex->vsm::VSMIndex + + + + +vsm::Vector + + +Vector + ++ + cosine_similarity(other: SELF): Float ++ + inc(obj: nullable Object) ++ + norm(): Float ++ + sorted_dimensions(): Array[nullable Object] ++ + tmp(): ArrayMap[nullable Object, Float] + + +vsm::FileIndex + + +FileIndex + ++ + accept_file(path: String): Bool ++ + index_dir(dir: String, auto_update: nullable Bool) ++ + index_file(path: String, auto_update: nullable Bool): nullable DOC ++ + index_files(paths: Collection[String], auto_update: nullable Bool) ++ + parse_file(file: String): Vector + + +vsm::FileIndex->vsm::StringIndex + + + + +vsm::Document + + +Document + + +