Friendly Sam is a software toolbox for optimization-based modeling and simulation

Friendly Sam is a toolbox developed to formulate and solve optimization-based models of energy systems, but it could be used for many other systems too. Friendly Sam is designed to produce readable and understandable model specifications. It is developed with the Python ecosystem of scientific tools in mind and can be used together with numpy, pandas, matplotlib and many of your other favorite tools.

Note

Friendly Sam is work in progress. Please post any questions or issues on the Issue Tracker.

Friendly Sam is friendly in a number of ways:

Flows of resources
The frien in friendly stands for flows of resources in energy system networks. With Friendly Sam, we model power plants, energy storages, consumers and other components as nodes in a network, interconnected by flows of “resources”. Resources is a common name for all the different flows you could model: district heating and cooling, electric power, fuels, etc.
User-friendly
Friendly Sam is user-friendly. Instead of a global namespace with variable names like VHSTOLOADT, we use object-oriented code with descriptive names like model[“Heat storage A”].accumulation(42). Your model becomes easier to write, understand and maintain.
Open source
Friendly Sam is open source software, because we think it’s friendly and smart to collaborate. Friendly Sam is released under LGPL v3 license. The source code is on GitHub.

Contents:

How to install Friendly Sam

Get Python 3

Friendly Sam is developed in Python 3 (at the time of this writing, Python 3.4). Download and install it now, if you haven’t already.

Use a virtual environment

It is highly recommended that you use a virtual environment. It’s not strictly necessary, but if you choose not to, there is a risk that you will have conflicts between different versions of the packages that Friendly Sam and other Python packages depend on. Google for python virtualenv if you want to learn more. If not, you can also do it the way I do, using vex.

  • If you are on Windows
    1. Open a command prompt.
    2. Make sure you have the latest setuptools by running pip install setuptools --upgrade
    3. Install vex by running pip install --user vex
    4. Create a virtual environment named my_project_name and enter it by running vex -m --python C:\Python34\python.exe my_project_name cmd

    Now, whenever you want to use your virtual environment, open a command prompt and run vex my_project_name cmd.

  • If you are on Linux

    Basically, you follow the instructions for Windows above but exchange C:\Python34\python.exe for something more suitable, and then do vex my_project_name bash instead. Also see the docs for vex if you have problems. Make sure your pip and python commands points to pip3 and python3 respectively, in case you have multiple versions of python installed.

If you get the error: distutils.errors.DistutilsOptionError: can't combine user with prefix when trying to install vex, execute pip with the –prefix flag: pip install --user --install-option="--prefix=" vex

Install Friendly Sam

Assuming you have entered/activated your Python virtual environment, or wherever you want to install it, open a command prompt/shell and run the command:

pip install friendlysam

Optional dependencies

If you want to add support for pandas related stuff, or for saving and loading models using dill, do one of:

pip install friendlysam[pandas]
pip install friendlysam[pickling]
pip install friendlysam[pandas,pickling]

For developers

Install in developer mode

If you are developing the source code of Friendly Sam, you probably want to install it in “develop” mode instead. This has two benefits. First, you get some extra dependencies such as nose (testing package), sphinx (documentation package) and twine and wheel (used for releasing), etc. Second, you won’t have to reinstall the package into your Python site-packages directory every time you change something.

  1. Get Python 3. (Note: If you are on Windows it might be convenient to use a ready-made distribution like WinPython and skip step 5 below, but we can’t guarantee it will work.)

  2. Download the source code

  3. You probably want to install Friendly Sam in a virtual environment. Create one and activate it before you take the next step.

  4. Now, to install Friendly Sam in develop mode, do this:

    pip install -r develop.txt
    

Note

If you are on Windows, pip-installation of some packages will fail if you don’t have a compiler correctly configured. One such example is NumPy. A simple way around it is to install binaries from Christoph Gohlke’s website for the packages that throw errors when you do pip install -r develop.txt.

Let’s say you are on Windows and download an installer called something like numpy-MKL-1.9.0.win-amd64-py3.4.exe. Don’t just run the file, because then it will be installed in your “main” Python installation (usually at C:\Python34). Instead, do this:

  1. Open a command prompt.

  2. Go into your virtual environment (e.g. vex my_project_name cmd).

  3. (option a) Do this if you have an .exe file:

    easy_install numpy-MKL-1.9.0.win-amd64-py3.4.exe
    
  1. (option b) Or, if you have a .whl file file, e.g. numpy-1.9.2+mkl-cp34-none-win_amd64.whl, do this:

    pip install numpy-1.9.2+mkl-cp34-none-win_amd64.whl
    

Make Sphinx documentation

The documentation for residues is made with Sphinx and hosted with Read the Docs. To parse nice, human-readable docstrings, we use Napoleon.

  • If you want to make a very minor change to the documentation, you can actually just edit the source, push to the github repository and magically, the docs will update at readthedocs.org.

  • However, if you want to edit the docs a lot, you probably want to make test builds on your own machine. In that case, you need to learn about Sphinx. To build the docs, open a command prompt, go to friendlysam\docs and run the command:

    make html
    

The resulting HTML can be previewed under friendlysam\docs\_build\index.html.

Run tests

Please run the tests before pushing to the master branch.

To run all the tests, including doctests in the source code and doctests in this documentation, go to the project root directory and run:

nosetests --with-doctest --doctest-options=+ELLIPSIS

Releasing Friendly Sam

Before releasing

  1. Make sure that the documentation is complete and builds OK. In friendlysam/docs:

    make html
    
  2. Make sure that the tests pass. In project root directory:

    nosetests --with-doctest --doctest-options=+ELLIPSIS
    
  3. Make sure the changelog is up-to-date.

  4. Update version.txt with the next semantic version number: http://semver.org/

  5. Commit and push to master branch. Then tag the release and push the tag:

    git tag -a vX.Y.Z -m 'Version X.Y.Z'
    git push origin vX.Y.Z
    

Releasing to PyPI

If Friendly Sam is installed in develop mode, you should already have twine (for secure communication with PyPI) and wheel (for building wheel distribution files).

  1. To put things on PyPI, you have to register on PyPI, and you should register on the test PyPI too:

  2. Make sure that your account is activated. You should get an email from PyPI.

  3. Make sure you are added as a maintainer of the friendlysam repository at PyPI/testPyPI.

  4. Create yourself a file called .pypirc and put it in your home directory. If you are on Windows, the file path should be``C:Usersyourusername.pypirc``. Put the following content in it:

    [distutils]
    index-servers =
        pypi
        test
    
    [pypi]
    repository:https://pypi.python.org/pypi
    username:your_pypi_username
    
    [test]
    repository:https://testpypi.python.org/pypi
    username:your_testpypi_username
    
  5. (Windows users) For Windows, there is a nice pypi.bat you can use.

    To register info about the package on PyPI, first push to the PyPI test site:

    pypi.bat register test
    

    You will be asked for your PyPI test password. Make sure it turned out as you wanted. Then do the real thing:

    pypi.bat register pypi
    

    To build and upload the distribution, do this:

    pypi.bat upload test
    

    Twine will upload to PyPI and ask you for username and password. Check on the test site that everything is OK. You can also run pip install ... from the test repo to be sure. Then upload the package to the real repo by running:

    pypi.bat upload pypi
    
  1. (Linux/Mac users) You can easily translate pypi.bat into a bash script. Please do so and contribute it to the repository!

What Friendly Sam is for

Why build another tool?

There are a lot of different tools for optimization-based modeling. Why in the world do we need another one?

The short answer is this: Friendly Sam is a domain specific toolbox. For the type of models we work with, the model code is shorter, more readable and easier to debug than it would be with many other tools. Furthermore, Friendly Sam makes data handling and analysis easier. Because Friendly Sam is implemented in Python, we get access to all our favorite Python tools for scientific computing and visualization, including Pandas, NumPy, SciPy, matplotlib, etc. This is a strong advantage because the majority of our modeling work is preparing input data and analyzing results.

In the coming paragraphs we’ll explain more about what Friendly Sam is. And at end we’ll also say a few things Friendly Sam is not.

Data handling is easier with Python

Friendly Sam was designed to simplify our work with optimization-based models of energy systems, so-called dispatch models. This is a common type of model in research and in applied analysis of energy systems, based on the thought that the operator(s) of an energy system always act so as to minimize the cost of delivering energy to customers, or maybe (in a parallel universe) to minimize the carbon emissions, or some other objective function. A dispatch model is usually formulated as a minimization problem: “Minimize the operation cost of this system in this time period, subject to the technical and legal constraints of the system.”

There are a zillion different variants of such models, but many of them have in common that there is a lot of data going in and out. Some examples of possible input data are prices for different forms of energy, demand profiles, technical constraints, etc. The output data could be operation decisions, system costs, greenhouse gas emissions, and many other things. Therefore, a large part of our modeling work is data handling: Reading and wrangling data files, transforming and resampling input and output data, visualizing results, making statistical tests, etc.

Many optimization-based models are implemented using a generic optimization modeling language like GAMS, AMPL, AIMMS or CMPL. These languages can be wonderful to work with when formulating models because they are made specifically for optimization, and they are efficient in transforming your human-readable code into something that can be understood by almost any optimization solver. However, the infrastructure for handling input and output data in GAMS and AMPL is sub-optimal (pun intended). Anyone who implemented a large, complicated model in one of those languages knows it’s not an easy ride to keep track of all the data going in and out, especially not if you want to make a lot of similar runs with different parameter sets. I know several people who wrote their own tools for getting inputs and outputs back and forth between GAMS and their favorite data crunching tool (Excel, Python, MATLAB, R, etc).

When we started writing what would later become Friendly Sam, we chose Python because of the great ecosystem of open source tools that come with it. We have paid specific attention to numpy, pandas, and matplotlib when developing Friendly Sam. It’s not necessary to use these tools with Friendly Sam, but there is a great chance they will make your life easier. What about optimization then? To formulate and solve the actual optimization problems, we first used the Python API of the Gurobi optimizer. Gurobi’s Python API exposes a Variable class with overloaded operators for addition, multiplication, etc, so you can make algebraic expressions for the optimization objective and all the constraints in Python code. The Gurobi backend then translates these expression objects into a well-formed optimization problem, solves the problem and delivers the solution back through the Python API so you never have to leave Python. In Friendly Sam 1.0 we have created an abstraction layer to reduce the dependence on a certain solver backend. We are now using PuLP to interact with the Gurobi and CBC solvers, but you never have to interact directly with the backend, and it is not too hard to switch to another backend if we want to.

Domain specific toolbox

Friendly Sam is a Python library for formulating, running, and analyzing optimization-based models of energy systems.

In fact, it’s not only suitable for modeling energy systems, but also for other systems where you want to optimize flow networks of physical or abstract quantities, be it energy carriers, money, solid waste, cargo deliveries, virtual water or something else.

In principle, you are not even restricted to modeling systems with flow networks, because the optimization engine behind Friendly Sam is exposed so you can formulate a large class of optimization problems. But if you want a generic tool for formulating optimization problems you should probably check out other tools instead. In Python it’s worth to look at CyLP, cvxpy, PuLP, and Pyomo. If you want a pure optimization language, look at GAMS, AMPL, AIMMS or CMPL.

So although Friendly Sam can be used as a rather generic optimization modeling tool, it is domain specific in the sense that it has vocabulary for energy systems and similar systems. We developed it specifically to help us formulate dispatch models. In our energy system models, there are almost always balance equations for energy or materials, so Friendly Sam contains definitions of things like FlowNetwork, Node and Cluster to simplify the formulation of such constraints. And the Node class is a perfect starting point for modeling things like power plants, energy storages, and other things you typically find in an energy system. Friendly Sam also has a simple formulation of a myopic dispatch model of the type we often encounter in the academic literature on energy system modeling. If you use these building blocks, you will have to think less about sign errors in balance equations and instead concentrate on what your model really means.

Friendly Sam code is meant to be readable. For example, in a district heating model we can have instances of Node subclasses, one named LinearCHP, another named HeatPump, etc. This makes perfect sense to us, because the code is naturally structured similar to how we think about the energy system we are modeling. When the underlying optimization problem is solved, we can query the state of the model objects with code like heat_pump.consumption['power'](time).

The code can also be easier to debug. When you have a bewildering error somewhere, it can be helpful to just eyeball the constraints of your optimization problem, to see if you can spot the error. Friendly Sam makes this easier by automatically naming constraints after their “owner”, for example the HeatPump instance we just mentioned. You can also name variables and add descriptions to constraints. These features help you understand where things come from when you are looking at a long list of constraints.

What Friendly Sam is not

First, we want to clarify that Friendly Sam is not “a model”. It is a toolbox we use to build models.

Second, Friendly Sam is not fool proof. It is entirely possible to make models that are stupid or wrong with Friendly Sam. We have tried to design Friendly Sam to produce readable, understandable, debuggable models, and to make idioms and conventions that help to avoid common errors. But having this tool is not an alternative to knowing and understanding the optimization problems you are creating. Friendly Sam is a tool to help us focus on what is important, rather than chasing indexing errors and how to formulate piecewise affine functions using special ordered sets.

Third, Friendly Sam is not primarily optimized for speed. If you want to solve a really big model fast, you are probably better off with something like AMPL or GAMS, or maybe writing your own code in a compiled language. However, if your model is moderately big you might get the job done faster with Friendly Sam because debugging, data handling, analysis and visualization will be so much faster. In our experience, the development phase often consumes more time and money than the computation phase, so development convenience is often more important than execution speed.

OK, let’s get started!

You are now ready to read about Variables and expressions.

Now that you know What Friendly Sam is for, let’s get started!

Variables and expressions

In Friendly Sam, each variable is an instance of the Variable class. Let’s create one:

>>> from friendlysam import Variable
>>> my_var = Variable('x')
>>> my_var
<friendlysam.opt.Variable at 0x...: x>
>>> print(my_var)
x

Variables can be added, multiplied, subtracted, and so on, to form expressions, including equalities and inequalities.

>>> expressions = [
...     my_var * 2 + 1,
...     (my_var + 1) * 2,
...     my_var * 2 <= 3
... ]
>>> for expr in expressions:
...     expr
<friendlysam.opt.Add at 0x...>
<friendlysam.opt.Mul at 0x...>
<friendlysam.opt.LessEqual at 0x...>
>>>
>>> for expr in expressions:
...     print(expr)
x * 2 + 1
(x + 1) * 2
x * 2 <= 3

Warning

The operator == is reserved for checking object similarity, just like we are used to in Python. To create the relation “x equals y”, use Eq:

>>> from friendlysam import Eq
>>> my_var == 1
False
>>> print(Eq(my_var, 1))
x == 1

There is also a nice Sum operation you should use for large sums. Using the built-in sum() will create a deeply nested and very inefficient tree of Add objects.

>>> from friendlysam import Sum
>>> many_terms = [my_var * i for i in range(100)]
>>> Sum(many_terms)
<friendlysam.opt.Sum at 0x...>
>>> sum(many_terms)
<friendlysam.opt.Add at 0x...>

Names don’t mean anything

In the example above, we named the Variable object 'x'. This is nothing more than a string attached to the object, and it does not say anything about the identity of the variable. In principle you can have several Variable objects with the same name, but that’s really confusing and should not be necessary.

>>> my_var = Variable('y')
>>> my_other_var = Variable('y')
>>> my_var == my_other_var
False
>>> print(my_var + my_other_var)
y + y

It is often a good idea to give your variables names you can recognize, because that simplifies debugging when you want to inspect the expressions you have made with the variables. But if you don’t want to name variables you don’t have to. The variables are then named automatically.

>>> Variable()
<friendlysam.opt.Variable at 0x...: x1>
>>> Variable()
<friendlysam.opt.Variable at 0x...: x2>

VariableCollection is like an indexed Variable

There is also a convenient class called VariableCollection. It is a sort of lazy dictionary, which creates variables when you ask for them:

>>> from friendlysam import VariableCollection
>>> z = VariableCollection('z')
>>> z
<friendlysam.opt.VariableCollection at 0x...: z>
>>> z(1)
<friendlysam.opt.Variable at 0x...: z(1)>
>>> z((1, 'a'))
<friendlysam.opt.Variable at 0x...: z((1, 'a'))>
>>> z(None)
<friendlysam.opt.Variable at 0x...: z(None)>

You can think of VariableCollection as an indexed variable, but all it really does is to create variables when you call it, and then remember them.

The index must be hashable. For example, tuples are valid indices, but not lists:

>>> z((3, 1, 4))
<friendlysam.opt.Variable at 0x...: z((3, 1, 4))>
>>> z([3, 1, 4])
Traceback (most recent call last):
...
TypeError: unhashable type: 'list'

Variables can be named in a namespace, like this:

>>> from friendlysam import namespace
>>> with namespace('cheese'):
...     cheese1 = Variable('gorgonzola')
...     cheese2 = VariableCollection('ricotta')
...
>>> cheese1
<friendlysam.opt.Variable at 0x...: cheese.gorgonzola>
>>> cheese2
<friendlysam.opt.VariableCollection at 0x...: cheese.ricotta>

The namespace doesn’t affect the function of a variable in any way. It only prepends a string representation of whatever object to the variable name, so you can also do things like this:

>>> with namespace(dict()):
...     Variable('x')
...
<friendlysam.opt.Variable at 0x...: {}.x>

Variables can have values

You can assign a value to a variable. The variable will still work in expressions:

>>> x = Variable('x')
>>> x.value = 39
>>> expression = x + 3
>>> expression
<friendlysam.opt.Add at 0x...>
>>> print(expression)
x + 3

The difference is that you can now evaluate expressions. But note that the expression object is unchanged.

>>> float(expression)
42.0
>>> print(expression)
x + 3

You can change or delete the value:

>>> x.value = 0.5
>>> int(expression)
3
>>> float(expression)
3.5
>>> expression.value
3.5
>>> del x.value
>>> float(expression)
Traceback (most recent call last):
...
friendlysam.opt.NoValueError: cannot get a numeric value: x + 3 evaluates to x + 3

And it works for relations, too:

>>> x.value = 10
>>> (x <= 12).value
True

Expressions are immutable

Expressions are hashed by structure: If they do the same thing, they hash and compare equal. This also means they are considered equal e.g. as dict keys.

>>> expr1 = x * 2
>>> expr2 = x * 2
>>> expr1 is expr2 # Different objects!
False
>>> expr1 == expr2 # But similar
True
>>> d = dict()
>>> d[expr1] = 'some value'
>>> d[expr2]
'some value'

Expressions are immutable, meaning that their state can never be changed. In the example above, expr1 == expr2 and that will always be true. Two expressions are interchangeable if (and only if) they compare equal. For any purpose, in any situation, expr1 will always do the same thing as expr2.

However, as you saw above, the result of float(expr1) may vary depending on whether variables in the expression have values. Let’s look a little bit closer:

>>> x.value = 3
>>> expression = x + 39
>>> float(expression)
42.0
>>> x.value = 100
>>> another_expression = x + 39
>>> expression == another_expression
True
>>> float(expression)
139.0
>>> float(another_expression)
139.0

This is pretty much analogous to a tuple of mutable objects. The tuple itself may never change, but its contents may:

>>> a = [1, 2, 3]
>>> my_tuple = (a, 'something')
>>> my_tuple
([1, 2, 3], 'something')
>>> a[:] = ['changed'] # Only changing the contents of the list
>>> another_tuple = (a, 'something')
>>> my_tuple == another_tuple
True
>>> my_tuple
(['changed'], 'something')
>>> another_tuple
(['changed'], 'something')

Behind value is evaluate()

You might want to know what is happening behind the scenes when you ask for expression.value or float(expression). In that case, check out the method evaluate().

Optimization problems

Creating a problem

We use Friendly Sam to formulate MILP problems. The optimization library could be extended to allow other types of problems, too, but this is what is supported today.

Now, let’s begin with a full example of an optimization problem.

>>> import friendlysam as fs
>>>
>>> # Create the problem
>>> x = fs.VariableCollection('x')
>>> prob = fs.Problem()
>>> prob.objective = fs.Maximize(x(1) + x(2))
>>> prob.add(8 * x(1) + 4 * x(2) <= 11)
>>> prob.add(2 * x(1) + 4 * x(2) <= 5)
>>>
>>> # Get a solver and solve the problem
>>> solver = fs.get_solver()
>>> solution = solver.solve(prob)
>>> type(solution)
<class 'dict'>
>>> solution[x(1)]
1.0
>>> solution[x(2)]
0.75

The solver does not in any way affect the problem or the variables. It just reads the problem, solves it and handles back a dict with your Variable objects as keys and their solutions as values.

If you set the value of some variables, those will be inserted into the problem before solving it:

>>> x(1).value = 0
>>> solution = solver.solve(prob)
>>> solution
{<friendlysam.opt.Variable at 0x...: x(2)>: 1.25}
>>> x(1) in solution
False

x(1) is not in the solution, because you already set its value, so it was handled like a number by the solver.

Debugging constraints

Now let’s add another constraint:

>>> x(1).value = 0
>>> prob.add(1 <= x(1))
>>> solver.solve(prob)
Traceback (most recent call last):
...
friendlysam.opt.ConstraintError: The expression in <Constraint: Ad hoc constraint> evaluates to False, so the problem is infeasible.

In this case it’s obvious why the problem could not be solved. But for argument’s sake, let’s say we didn’t know which constraint was causing a problem. The error message was not too helpful, but the ConstraintError luckily also contains a reference to the constraint that failed, so we can pick it out like this:

>>> try:
...     solver.solve(prob)
... except fs.ConstraintError as e:
...     failed_constraint = e.constraint
...     print(repr(failed_constraint))
...     print(repr(failed_constraint.expr))
...     print(failed_constraint.expr)
...     print(failed_constraint.desc)
...     print(failed_constraint.origin)
...
<friendlysam.opt.Constraint at 0x...>
<friendlysam.opt.LessEqual at 0x...>
1 <= x(1)
Ad hoc constraint
None

OK, that’s helpful! We got the problematic constraint out. And there are a few things you should note.

  1. The type of the failed constraint is friendlysam.opt.Constraint. It was automatically created when we added a friendlysam.opt.LessEqual constraint to the problem, and its sole purpose is to wrap the inequality 1 <= x(1) and to add some metadata.
  2. The Constraint object contains the LessEqual object that we added to the problem.
  3. The Constraint object contains also a description desc and a variable called origin which is supposed to say something about where the constraint comes from.

Note

There is a quicker way of printing out some info about a constraint: long_description:

>>> print(failed_constraint.long_description)
<friendlysam.opt.Constraint at 0x...>
Description: Ad hoc constraint
Origin: None

If you want to make your model easier to debug, you can use Constraint instances with custom description and/or origin, like in this stupid example:

>>> from friendlysam import Constraint
>>> def constr(var, parameter):
...     return var / 42 >= parameter
>>> for i in range(5):
...     expr = constr(x(i), i)
...     origin = (constr, x(i), i)
...     prob += Constraint(expr, desc='Some description', origin=origin)
...

Different ways to add constraints

Note

In the examples above, we added constraints like this:

>>> prob.add(8 * x(1) + 4 * x(2) <= 11)
>>> prob += Constraint(expr, desc='Some description', origin=origin)

These two methods are equivalent, so just choose the syntax you like best.

You can also send an iterable (even a generator), and the items in the iterable can also be iterables, e.g:

>>> prob += ([constr(x(i), i), constr(x(i+1), i)] for i in range(5))

See the documentation for add() for all the details.

Special ordered sets

Friendly Sam also supports special ordered sets. You specify them as a sort of constraint: Check out SOS1 and SOS2.

Model basics: Parts and constraints

Interconnected parts

Friendly Sam is made for optimization-based modeling of interconnected parts, producing and consuming various resources. For example, an urban energy system may have grids for district heating, electric power, fossil gas, etc. Consumers, producers, storages and other parts are connected to each other through these grids. To describe these relations, Friendly Sam models make heavy use of the Part class and its subclasses like Node, FlowNetwork, Cluster, Storage, etc. We will introduce these in due time, but first a few general things about Part.

Parts have indexed constraints

A Part typically represents something physical, like a heat consumer or a power grid. You can attach constraint functions to parts. Constraint functions are probably easiest to explain with a concrete example:

>>> from friendlysam import Part, namespace, VariableCollection
>>> class ChocolateFactory(Part):
...     def __init__(self):
...         with namespace(self):
...             self.output = VariableCollection('output')
...         self.constraints += self.more_than_last
...
...     def more_than_last(self, time):
...         last = self.step_time(time, -1)
...         return self.output(last) <= self.output(time)
...

OK, what happens above is the following: We define ChocolateFactory as a subclass of Part. Upon setup, in __init__(), we add a constraint function called more_than_last, which defines the (admittedly bizarre) rule that the factory may never decrease its output from one time step to the next.

In Friendly Sam’s vocabulary, the time argument in the example above is called an index. Our typical use case for indexing is a discrete time model, where each hour, day, year, or whatever time period, is an index of the model, and each constraint “belongs” to a time step just like in the silly example above.

Going back to the example, we can get the constraints out by making a ChocolateFactory instance and calling constraints.make() with an index:

>>> chocolate_factory = ChocolateFactory() # Create an instance
>>> constraints = chocolate_factory.constraints.make(47)
>>> constraints
{<friendlysam.opt.Constraint at 0x...>}
>>> for c in constraints:
...     print(c.expr)
...
ChocolateFactory0001.output(46) <= ChocolateFactory0001.output(47)

The result of constraints.make(47) is a set with one single constraint in it, saying that output at “time” 47 must be greater than or equal to output at “time” 46.

Note

In the example above, we wrote last = self.step_time(time, -1) instead of just last = t-1. This is because Part has a bunch of nice functions to help out with time indexing. Read the API documentation for step_time(). Also check out times() and times_between(). For example, because we used step_time(...), we can easily change the time representation of our chocolate factory like this:

>>> from pandas import Timestamp, Timedelta
>>> chocolate_factory.time_unit = Timedelta('6h')
>>> constraints = chocolate_factory.constraints.make(Timestamp('2015-06-10 18:00'))
>>> for c in constraints:
...     print(c.expr)
...
ChocolateFactory0001.output(2015-06-10 12:00:00) <= ChocolateFactory0001.output(2015-06-10 18:00:00)

Advanced indexing

Indexing is really just a way to organize constraints in groups that belong together. When we have a whole bunch of parts, to get all the constraints that belong together, we can do things like this:

>>> from itertools import chain
>>> parts = Part(), Part(), Part() # Put something more useful here...
>>> some_index = 'could be anything'
>>> constraints = set.union(*(p.constraints.make(some_index) for p in parts))

Indexing is typically used to represent time, but it is really up to you to decide what an index means, and what to use as indices. We call it “index” rather than “time” because it is something more general than a representation of time. In fact, any hashable object can be used as an index, so you can do all sorts of complicated things. Examples of indexing can be found in the docs for step_time(). Also check out times() and times_between().

Friendly Sam currently has no mechanism for using constraint functions without indices. If you want to make a static model and really don’t need indexing, then just use some common index like None or 0 for everything. (Or come up with a better solution and discuss it with us on GitHub.)

In the next section Flow networks: Nodes and resources you will also see how indexing is used to represent time in flow networks.

Flow networks: Nodes and resources

Note

This tutorial does not cover everything. To learn more, follow the links into the API reference for Node, FlowNetwork, Cluster etc.

Friendly Sam makes it easy to formulate optimization problems with flow networks. Let’s begin with an example.

Nodes and balance constraints

An example

Custom types of nodes should typically be created by subclassing Node, like this:

>>> from friendlysam import Node, VariableCollection, namespace
>>> class PowerPlant(Node):
...     def __init__(self):
...         with namespace(self):
...             x = VariableCollection('output')
...         self.production['power'] = x
...
>>> class Consumer(Node):
...     def __init__(self, demand):
...         self.consumption['power'] = lambda time: demand[time]
...

We have now defined a PowerPlant class inheriting Node, and a Consumer class, also inheriting Node. The power plant has its production['power'] equal to a VariableCollection, and the consumer has consumption['power'] equal to the value found in the argument demand. Let’s create instances and test them:

>>> power_plant = PowerPlant()
>>> power_plant.production['power'](3)
<friendlysam.opt.Variable at 0x...: PowerPlant0001.output(3)>
>>> power_demand = [25, 30, 33, 29, 27]
>>> consumer = Consumer(power_demand)
>>> consumer.consumption['power'](3)
29

Now connect the two nodes:

>>> from friendlysam import FlowNetwork
>>> power_grid = FlowNetwork('power', name='Power grid')
>>> power_grid.connect(power_plant, consumer)
>>> power_grid.children == {power_plant, consumer}
True

The Consumer instance and the PowerPlant instance were added to the power grid, and can now be found as children of the FlowNetwork.

Note

In this example, we use the key 'power' in a few different places. Whatever we put as a key in a production or consumption dictionary, or a similar place, is called a resource. You are not limited to strings like 'power' but could use any hashable type: numbers, tuples, most other objects, etc.

Balance constraints

Each Node has a pre-defined constraint function for balance constraints, so calling constraints.make() on the nodes creates balance constraints. The dictionaries production and consumption are automatically included in these balance constraints. The connect() call creates a flow between two nodes, and it adds this flow to the appropriate outflows or inflows on those two nodes. Each Node can then formulate its own balance constraints:

>>> for part in [consumer, power_plant, power_grid]:
...     for constraint in part.constraints.make(3):
...         print(constraint.long_description)
...         print(constraint.expr)
...         print()
...
<friendlysam.opt.Constraint at 0x...>
Description: Balance constraint (resource=power)
Origin: CallTo(func=<bound method Consumer.balance_constraints of <Consumer at 0x...: Consumer0001>>, index=3, owner=<Consumer at 0x...: Consumer0001>)
Power grid.flow(PowerPlant0001-->Consumer0001)(3) == 29

<friendlysam.opt.Constraint at 0x...>
Description: Balance constraint (resource=power)
Origin: CallTo(func=<bound method PowerPlant.balance_constraints of ...>, index=3, owner=<PowerPlant at 0x...: PowerPlant0001>)
PowerPlant0001.output(3) == Power grid.flow(PowerPlant0001-->Consumer0001)(3)

How balance constraints are made

Here are a few simple rules for how balance constraints are made:

  • Each Node has the five dictionaries consumption, production, accumulation, inflows, and outflows.

  • Whatever you decide to put as a key in any of these dictionaries is called a resource.

  • For each resource present in any of the dictionaries, the Node produces balance constraints like this:

    (sum of inflows) + production = consumption + accumulation + (sum of outflows)

  • The constraints of the node are accessed by calling something like

    >>> index = 3
    >>> constraints = power_plant.constraints.make(index)
    

    The index is passed on to the functions: production[resource](index), consumption[resource](index), etc. In this way, indices always represent time when you are working with nodes and flow networks. You can use any function or object as production[resource], consumption[resource], etc, as long as it is callable.

Note

A Node instance will always produce balance constraints for each of its resources. Let’s say we had not connected the PowerPlant instance to the consumer, then its balance constraint would be PowerPlant0001.output(3) == 0. (Try it yourself!) In other words, flows of resources must always be balanced in a Friendly Sam model. Noone may produce a resource like 'power' if it has nowhere to go, and noone can consume it unless there is a source.

Custom names

Note

You can name your Node instances if you want something more personal than PowerPlant0001. Just set the property name, for example in the __init__ function, like this:

>>> class CHPPlant(Node):
...     def __init__(self, name=None):
...         if name:
...             self.name = name
...         ...
>>> chp_plant = CHPPlant(name='Rya KVV')
>>> chp_plant.name == str(chp_plant) == 'Rya KVV'
True

FlowNetwork

A FlowNetwork essentially does two things: It creates the variable collections representing flows in the network, and it modifies the inflows and outflows of nodes when you call connect().

Unidirectional by default

Connections are unidirectional, so when you connect(node1, node2) things can flow from node1 to node2. Make the opposite connection if you want a bidirectional flow, or use this shorthand:

>>> power_grid.connect(power_plant, consumer, bidirectional=True)

Flow restrictions

To limit the flow between two nodes, get the flow VariableCollection and set its upper bound ub:

>>> flow = power_grid.get_flow(power_plant, consumer)
>>> flow
<friendlysam.opt.VariableCollection at 0x...: Power grid.flow(PowerPlant0001-->Consumer0001)>
>>> flow.ub = 40

Clusters and multi-area models

A cluster is fully connected

Sometimes we are not interested in making a full network model specifying all the flows between different nodes. The Cluster class is a handy type of Node for that. It is a type of node that can contain other nodes, and it essentially acts like a fully connected network, where all nodes are connected to all others.

When a Node is put in a Cluster, the child Node will no longer make balance constraints, and instead the Cluster creates an aggregated balance constraint, summing up the production, consumption and accumulation of its contained children.

>>> from friendlysam import Cluster
>>> power_plant = PowerPlant()
>>> consumer = Consumer(power_demand)
>>> power_cluster = Cluster(power_plant, consumer, resource='power', name='Power cluster')
>>> for part in power_cluster.descendants_and_self:
...     for constraint in part.constraints.make(2):
...         print(constraint.long_description)
...         print(constraint.expr)
...
<friendlysam.opt.Constraint at 0x...>
Description: Balance constraint (resource=power)
Origin: CallTo(func=<bound method Cluster.balance_constraints ...>, index=2, owner=<Cluster at 0x...: Power cluster>)
PowerPlant0002.output(2) == 33

Multi-area models

A Cluster instance can be used like any other Node, for example in a FlowNetwork. This is a simple way of making a multi-area model of, say, a district heating system. Let’s say the system has a few areas with significant flow restrictions between them. Then create a flow network with interconnected clusters, something like this:

area_A = Cluster(*nodes_in_area_A, resource='heat')
area_B = Cluster(*nodes_in_area_B, resource='heat')
area_C = Cluster(*nodes_in_area_C, resource='heat')

heat_grid = FlowNetwork('heat')
heat_grid.connect(area_A, area_B, bidirectional=True, capacity=ab)
heat_grid.connect(area_A, area_C, bidirectional=True, capacity=ac)
heat_grid.connect(area_B, area_C, bidirectional=True, capacity=bc)

Time in flow networks

It is natural to think of indices like time periods: All the expressions for flows, production and consumption must add up, for each index (time period). As shown in the examples above, the balance constraints for an index is called by passing the index to production, consumption, outflows and inflows.

There is another dictionary which is always used in balance constraints: accumulation. It works just like the dictionaries production and consumption. To learn more, read the API docs for Storage, and look at this example:

>>> from friendlysam import Storage
>>> from pandas import Timestamp, Timedelta
>>> battery = Storage('power', name='Battery')
>>> battery.time_unit = Timedelta('3h')
>>> t = Timestamp('2015-06-10 18:00')
>>> print(battery.accumulation['power'](t))
Battery.volume(2015-06-10 21:00:00) - Battery.volume(2015-06-10 18:00:00)

Example model

To get a feeling for what’s possible with Friendly Sam, have a look at this example model:

https://github.com/sp-etx/example-model

API reference

Variables

Variable([name, lb, ub, domain]) A variable to build expressions with.
VariableCollection([name]) A lazy collection of Variable instances.
Domain Domain of a variable.
namespace(name) Prefix variable names.

Expressions

Operation An operation on some arguments.
Add Addition operator.
Sub Subtraction operator.
Mul Subtraction operator.
Sum A sum of items.
dot(a, b) Make expression for the scalar product of two vectors.
Relation Base class for binary relations.
Less The relation “less than”.
LessEqual The relation “less than or equal to”.
Eq The relation “equals”.

Constraints and optimization

get_solver([engine, options]) Get a solver object.
Problem() An optimization problem.
Maximize(expr) A maximization objective.
Minimize(expr) A minimization objective.
Constraint(expr[, desc, origin]) An equality or inequality constraint.
SOS1(variables, **kwargs) Special ordered set, type 1
SOS2(variables, **kwargs) Special ordered set, type 2
piecewise_affine(points[, name]) Create a piecewise affine expression and constraints.
piecewise_affine_constraints(variables[, ...]) Constrains for a piecewise affine expression.

Models

Part([name]) A part of a model.
Node([name]) A node with balance constraints.
FlowNetwork(resource[, name]) Manages flows between nodes.
Cluster(*parts[, resource, name]) A node containing other nodes, fully connected.
Storage(resource[, capacity, maxchange, name]) Simple storage model.
ConstraintCollection(owner) Generates constraints from functions.
MyopicDispatchModel([t0, horizon, step, ...]) docstring for MyopicDispatchModel

Utilities

get_list(func, indices) Get a list of function values at indices.
get_series(func, indices, **kwargs) Get a pandas Series of function values at indices.

Exceptions

ConstraintError(*args[, constraint]) Raised when there is something wrong with a Constraint.
NoValueError Raised when a variable or expression has no value.
SolverError A generic exception raised by a solver instance.
InsanityError Raised when a sanity check fails.

Indices and tables