Friendly Sam is a software toolbox for optimization-based modeling and simulation¶
Friendly Sam is a toolbox developed to formulate and solve optimization-based models of energy systems, but it could be used for many other systems too. Friendly Sam is designed to produce readable and understandable model specifications. It is developed with the Python ecosystem of scientific tools in mind and can be used together with numpy, pandas, matplotlib and many of your other favorite tools.
Note
Friendly Sam is work in progress. Please post any questions or issues on the Issue Tracker.
Friendly Sam is friendly in a number of ways:¶
- Flows of resources
- The frien in friendly stands for flows of resources in energy system networks. With Friendly Sam, we model power plants, energy storages, consumers and other components as nodes in a network, interconnected by flows of “resources”. Resources is a common name for all the different flows you could model: district heating and cooling, electric power, fuels, etc.
- User-friendly
- Friendly Sam is user-friendly. Instead of a global namespace with variable names like
VHSTOLOADT
, we use object-oriented code with descriptive names likemodel[“Heat storage A”].accumulation(42)
. Your model becomes easier to write, understand and maintain. - Open source
- Friendly Sam is open source software, because we think it’s friendly and smart to collaborate. Friendly Sam is released under LGPL v3 license. The source code is on GitHub.
Contents:¶
How to install Friendly Sam¶
Get Python 3¶
Friendly Sam is developed in Python 3 (at the time of this writing, Python 3.4). Download and install it now, if you haven’t already.
Use a virtual environment¶
It is highly recommended that you use a virtual environment. It’s not strictly necessary, but if you choose not to, there is a risk that you will have conflicts between different versions of the packages that Friendly Sam and other Python packages depend on. Google for python virtualenv
if you want to learn more. If not, you can also do it the way I do, using vex.
- If you are on Windows
- Open a command prompt.
- Make sure you have the latest
setuptools
by runningpip install setuptools --upgrade
- Install vex by running
pip install --user vex
- Create a virtual environment named
my_project_name
and enter it by runningvex -m --python C:\Python34\python.exe my_project_name cmd
Now, whenever you want to use your virtual environment, open a command prompt and run
vex my_project_name cmd
.
If you are on Linux
Basically, you follow the instructions for Windows above but exchange
C:\Python34\python.exe
for something more suitable, and then dovex my_project_name bash
instead. Also see the docs for vex if you have problems.
Install Friendly Sam¶
Assuming you have entered/activated your Python virtual environment, or wherever you want to install it, open a command prompt/shell and run the command:
pip install friendlysam
Optional dependencies¶
If you want to add support for pandas
related stuff, or for saving and loading models using dill
, do one of:
pip install friendlysam[pandas]
pip install friendlysam[pickling]
pip install friendlysam[pandas,pickling]
For developers¶
Install in developer mode¶
If you are developing the source code of Friendly Sam, you probably want to install it in “develop” mode instead. This has two benefits. First, you get some extra dependencies such as nose
(testing package), sphinx
(documentation package) and twine
and wheel
(used for releasing), etc. Second, you won’t have to reinstall the package into your Python site-packages directory every time you change something.
Get Python 3. (Note: If you are on Windows it might be convenient to use a ready-made distribution like WinPython and skip step 5 below, but we can’t guarantee it will work.)
Download the source code
Alternative 1: Download a zip file: https://github.com/sp-etx/friendlysam/archive/master.zip
Alternative 2: If you know git, clone into the repository:
git clone https://github.com/sp-etx/friendlysam.git
You probably want to install Friendly Sam in a virtual environment. Create one and activate it before you take the next step.
Now, to install Friendly Sam in develop mode, do this:
pip install -r develop.txt
Note
If you are on Windows, pip
-installation of some packages will fail if you don’t have a compiler correctly configured. One such example is NumPy. A simple way around it is to install binaries from Christoph Gohlke’s website for the packages that throw errors when you do pip install -r develop.txt
.
Let’s say you are on Windows and download an installer called something like numpy-MKL-1.9.0.win-amd64-py3.4.exe
. Don’t just run the file, because then it will be installed in your “main” Python installation (usually at C:\Python34
). Instead, do this:
Open a command prompt.
Go into your virtual environment (e.g.
vex my_project_name cmd
).(option a) Do this if you have an .exe file:
easy_install numpy-MKL-1.9.0.win-amd64-py3.4.exe
(option b) Or, if you have a .whl file file, e.g.
numpy-1.9.2+mkl-cp34-none-win_amd64.whl
, do this:pip install numpy-1.9.2+mkl-cp34-none-win_amd64.whl
Make Sphinx documentation¶
The documentation for residues is made with Sphinx and hosted with Read the Docs. To parse nice, human-readable docstrings, we use Napoleon.
If you want to make a very minor change to the documentation, you can actually just edit the source, push to the github repository and magically, the docs will update at readthedocs.org.
However, if you want to edit the docs a lot, you probably want to make test builds on your own machine. In that case, you need to learn about Sphinx. To build the docs, open a command prompt, go to
friendlysam\docs
and run the command:make html
The resulting HTML can be previewed under friendlysam\docs\_build\index.html
.
Run tests¶
Please run the tests before pushing to the master branch.
To run all the tests, including doctests in the source code and doctests in this documentation, go to the project root directory and run:
nosetests --with-doctest --doctest-options=+ELLIPSIS
Releasing Friendly Sam¶
Before releasing¶
Make sure that the documentation is complete and builds OK. In
friendlysam/docs
:make htmlMake sure that the tests pass. In project root directory:
nosetests --with-doctest --doctest-options=+ELLIPSISMake sure the changelog is up-to-date.
Update
version.txt
with the next semantic version number: http://semver.org/Commit and push to
master
branch. Then tag the release and push the tag:git tag -a vX.Y.Z -m 'Version X.Y.Z' git push origin vX.Y.Z
Releasing to PyPI¶
If Friendly Sam is installed in develop mode, you should already have twine (for secure communication with PyPI) and wheel (for building wheel distribution files).
To put things on PyPI, you have to register on PyPI, and you should register on the test PyPI too:
Make sure that your account is activated. You should get an email from PyPI.
Make sure you are added as a maintainer of the friendlysam repository at PyPI/testPyPI.
Create yourself a file called
.pypirc
and put it in your home directory. If you are on Windows, the file path should be``C:Usersyourusername.pypirc``. Put the following content in it:[distutils] index-servers = pypi test [pypi] repository:https://pypi.python.org/pypi username:your_pypi_username [test] repository:https://testpypi.python.org/pypi username:your_testpypi_username(Windows users) For Windows, there is a nice
pypi.bat
you can use.To register info about the package on PyPI, first push to the PyPI test site:
pypi.bat register testYou will be asked for your PyPI test password. Make sure it turned out as you wanted. Then do the real thing:
pypi.bat register pypiTo build and upload the distribution, do this:
pypi.bat upload testTwine will upload to PyPI and ask you for username and password. Check on the test site that everything is OK. You can also run
pip install ...
from the test repo to be sure. Then upload the package to the real repo by running:pypi.bat upload pypi
- (Linux/Mac users) You can easily translate
pypi.bat
into a bash script. Please do so and contribute it to the repository!
What Friendly Sam is for¶
Why build another tool?¶
There are a lot of different tools for optimization-based modeling. Why in the world do we need another one?
The short answer is this: Friendly Sam is a domain specific toolbox. For the type of models we work with, the model code is shorter, more readable and easier to debug than it would be with many other tools. Furthermore, Friendly Sam makes data handling and analysis easier. Because Friendly Sam is implemented in Python, we get access to all our favorite Python tools for scientific computing and visualization, including Pandas, NumPy, SciPy, matplotlib, etc. This is a strong advantage because the majority of our modeling work is preparing input data and analyzing results.
In the coming paragraphs we’ll explain more about what Friendly Sam is. And at end we’ll also say a few things Friendly Sam is not.
Data handling is easier with Python¶
Friendly Sam was designed to simplify our work with optimization-based models of energy systems, so-called dispatch models. This is a common type of model in research and in applied analysis of energy systems, based on the thought that the operator(s) of an energy system always act so as to minimize the cost of delivering energy to customers, or maybe (in a parallel universe) to minimize the carbon emissions, or some other objective function. A dispatch model is usually formulated as a minimization problem: “Minimize the operation cost of this system in this time period, subject to the technical and legal constraints of the system.”
There are a zillion different variants of such models, but many of them have in common that there is a lot of data going in and out. Some examples of possible input data are prices for different forms of energy, demand profiles, technical constraints, etc. The output data could be operation decisions, system costs, greenhouse gas emissions, and many other things. Therefore, a large part of our modeling work is data handling: Reading and wrangling data files, transforming and resampling input and output data, visualizing results, making statistical tests, etc.
Many optimization-based models are implemented using a generic optimization modeling language like GAMS, AMPL, AIMMS or CMPL. These languages can be wonderful to work with when formulating models because they are made specifically for optimization, and they are efficient in transforming your human-readable code into something that can be understood by almost any optimization solver. However, the infrastructure for handling input and output data in GAMS and AMPL is sub-optimal (pun intended). Anyone who implemented a large, complicated model in one of those languages knows it’s not an easy ride to keep track of all the data going in and out, especially not if you want to make a lot of similar runs with different parameter sets. I know several people who wrote their own tools for getting inputs and outputs back and forth between GAMS and their favorite data crunching tool (Excel, Python, MATLAB, R, etc).
When we started writing what would later become Friendly Sam, we chose Python because of the great ecosystem of open source tools that come with it. We have paid specific attention to numpy, pandas, and matplotlib when developing Friendly Sam. It’s not necessary to use these tools with Friendly Sam, but there is a great chance they will make your life easier. What about optimization then? To formulate and solve the actual optimization problems, we first used the Python API of the Gurobi optimizer. Gurobi’s Python API exposes a Variable class with overloaded operators for addition, multiplication, etc, so you can make algebraic expressions for the optimization objective and all the constraints in Python code. The Gurobi backend then translates these expression objects into a well-formed optimization problem, solves the problem and delivers the solution back through the Python API so you never have to leave Python. In Friendly Sam 1.0 we have created an abstraction layer to reduce the dependence on a certain solver backend. We are now using PuLP to interact with the Gurobi and CBC solvers, but you never have to interact directly with the backend, and it is not too hard to switch to another backend if we want to.
Domain specific toolbox¶
Friendly Sam is a Python library for formulating, running, and analyzing optimization-based models of energy systems.
In fact, it’s not only suitable for modeling energy systems, but also for other systems where you want to optimize flow networks of physical or abstract quantities, be it energy carriers, money, solid waste, cargo deliveries, virtual water or something else.
In principle, you are not even restricted to modeling systems with flow networks, because the optimization engine behind Friendly Sam is exposed so you can formulate a large class of optimization problems. But if you want a generic tool for formulating optimization problems you should probably check out other tools instead. In Python it’s worth to look at CyLP, cvxpy, PuLP, and Pyomo. If you want a pure optimization language, look at GAMS, AMPL, AIMMS or CMPL.
So although Friendly Sam can be used as a rather generic optimization modeling tool, it is domain specific in the sense that it has vocabulary for energy systems and similar systems. We developed it specifically to help us formulate dispatch models. In our energy system models, there are almost always balance equations for energy or materials, so Friendly Sam contains definitions of things like FlowNetwork
, Node
and Cluster
to simplify the formulation of such constraints. And the Node
class is a perfect starting point for modeling things like power plants, energy storages, and other things you typically find in an energy system. Friendly Sam also has a simple formulation of a myopic dispatch model of the type we often encounter in the academic literature on energy system modeling. If you use these building blocks, you will have to think less about sign errors in balance equations and instead concentrate on what your model really means.
Friendly Sam code is meant to be readable. For example, in a district heating model we can have instances of Node
subclasses, one named LinearCHP
, another named HeatPump
, etc. This makes perfect sense to us, because the code is naturally structured similar to how we think about the energy system we are modeling. When the underlying optimization problem is solved, we can query the state of the model objects with code like heat_pump.consumption['power'](time)
.
The code can also be easier to debug. When you have a bewildering error somewhere, it can be helpful to just eyeball the constraints of your optimization problem, to see if you can spot the error. Friendly Sam makes this easier by automatically naming constraints after their “owner”, for example the HeatPump
instance we just mentioned. You can also name variables and add descriptions to constraints. These features help you understand where things come from when you are looking at a long list of constraints.
What Friendly Sam is not¶
First, we want to clarify that Friendly Sam is not “a model”. It is a toolbox we use to build models.
Second, Friendly Sam is not fool proof. It is entirely possible to make models that are stupid or wrong with Friendly Sam. We have tried to design Friendly Sam to produce readable, understandable, debuggable models, and to make idioms and conventions that help to avoid common errors. But having this tool is not an alternative to knowing and understanding the optimization problems you are creating. Friendly Sam is a tool to help us focus on what is important, rather than chasing indexing errors and how to formulate piecewise affine functions using special ordered sets.
Third, Friendly Sam is not primarily optimized for speed. If you want to solve a really big model fast, you are probably better off with something like AMPL or GAMS, or maybe writing your own code in a compiled language. However, if your model is moderately big you might get the job done faster with Friendly Sam because debugging, data handling, analysis and visualization will be so much faster. In our experience, the development phase often consumes more time and money than the computation phase, so development convenience is often more important than execution speed.
OK, let’s get started!¶
You are now ready to read about Variables and expressions.
Now that you know What Friendly Sam is for, let’s get started!
Variables and expressions¶
In Friendly Sam, each variable is an instance of the Variable
class. Let’s create one:
>>> from friendlysam import Variable
>>> my_var = Variable('x')
>>> my_var
<friendlysam.opt.Variable at 0x...: x>
>>> print(my_var)
x
Variables can be added, multiplied, subtracted, and so on, to form expressions, including equalities and inequalities.
>>> expressions = [
... my_var * 2 + 1,
... (my_var + 1) * 2,
... my_var * 2 <= 3
... ]
>>> for expr in expressions:
... expr
<friendlysam.opt.Add at 0x...>
<friendlysam.opt.Mul at 0x...>
<friendlysam.opt.LessEqual at 0x...>
>>>
>>> for expr in expressions:
... print(expr)
x * 2 + 1
(x + 1) * 2
x * 2 <= 3
Warning
The operator ==
is reserved for checking object similarity, just like we are used to in Python. To create the relation “x equals y”, use Eq
:
>>> from friendlysam import Eq
>>> my_var == 1
False
>>> print(Eq(my_var, 1))
x == 1
There is also a nice Sum
operation you should use for large sums. Using the built-in sum()
will create a deeply nested and very inefficient tree of Add
objects.
>>> from friendlysam import Sum
>>> many_terms = [my_var * i for i in range(100)]
>>> Sum(many_terms)
<friendlysam.opt.Sum at 0x...>
>>> sum(many_terms)
<friendlysam.opt.Add at 0x...>
Names don’t mean anything¶
In the example above, we named the Variable
object 'x'
. This is nothing more than a string attached to the object, and it does not say anything about the identity of the variable. In principle you can have several Variable
objects with the same name, but that’s really confusing and should not be necessary.
>>> my_var = Variable('y')
>>> my_other_var = Variable('y')
>>> my_var == my_other_var
False
>>> print(my_var + my_other_var)
y + y
It is often a good idea to give your variables names you can recognize, because that simplifies debugging when you want to inspect the expressions you have made with the variables. But if you don’t want to name variables you don’t have to. The variables are then named automatically.
>>> Variable()
<friendlysam.opt.Variable at 0x...: x1>
>>> Variable()
<friendlysam.opt.Variable at 0x...: x2>
VariableCollection is like an indexed Variable¶
There is also a convenient class called VariableCollection
. It is a sort of lazy dictionary, which creates variables when you ask for them:
>>> from friendlysam import VariableCollection
>>> z = VariableCollection('z')
>>> z
<friendlysam.opt.VariableCollection at 0x...: z>
>>> z(1)
<friendlysam.opt.Variable at 0x...: z(1)>
>>> z((1, 'a'))
<friendlysam.opt.Variable at 0x...: z((1, 'a'))>
>>> z(None)
<friendlysam.opt.Variable at 0x...: z(None)>
You can think of VariableCollection
as an indexed variable, but all it really does is to create variables when you call it, and then remember them.
The index must be hashable. For example, tuples are valid indices, but not lists:
>>> z((3, 1, 4))
<friendlysam.opt.Variable at 0x...: z((3, 1, 4))>
>>> z([3, 1, 4])
Traceback (most recent call last):
...
TypeError: unhashable type: 'list'
Variables can be named in a namespace, like this:
>>> from friendlysam import namespace
>>> with namespace('cheese'):
... cheese1 = Variable('gorgonzola')
... cheese2 = VariableCollection('ricotta')
...
>>> cheese1
<friendlysam.opt.Variable at 0x...: cheese.gorgonzola>
>>> cheese2
<friendlysam.opt.VariableCollection at 0x...: cheese.ricotta>
The namespace doesn’t affect the function of a variable in any way. It only prepends a string representation of whatever object to the variable name, so you can also do things like this:
>>> with namespace(dict()):
... Variable('x')
...
<friendlysam.opt.Variable at 0x...: {}.x>
Variables can have values¶
You can assign a value to a variable. The variable will still work in expressions:
>>> x = Variable('x')
>>> x.value = 39
>>> expression = x + 3
>>> expression
<friendlysam.opt.Add at 0x...>
>>> print(expression)
x + 3
The difference is that you can now evaluate expressions. But note that the expression object is unchanged.
>>> float(expression)
42.0
>>> print(expression)
x + 3
You can change or delete the value:
>>> x.value = 0.5
>>> int(expression)
3
>>> float(expression)
3.5
>>> expression.value
3.5
>>> del x.value
>>> float(expression)
Traceback (most recent call last):
...
friendlysam.opt.NoValueError: cannot get a numeric value: x + 3 evaluates to x + 3
And it works for relations, too:
>>> x.value = 10
>>> (x <= 12).value
True
Expressions are immutable¶
Expressions are hashed by structure: If they do the same thing, they hash and compare equal. This also means they are considered equal e.g. as dict
keys.
>>> expr1 = x * 2
>>> expr2 = x * 2
>>> expr1 is expr2 # Different objects!
False
>>> expr1 == expr2 # But similar
True
>>> d = dict()
>>> d[expr1] = 'some value'
>>> d[expr2]
'some value'
Expressions are immutable, meaning that their state can never be changed. In the example above, expr1 == expr2
and that will always be true. Two expressions are interchangeable if (and only if) they compare equal. For any purpose, in any situation, expr1
will always do the same thing as expr2
.
However, as you saw above, the result of float(expr1)
may vary depending on whether variables in the expression have values. Let’s look a little bit closer:
>>> x.value = 3
>>> expression = x + 39
>>> float(expression)
42.0
>>> x.value = 100
>>> another_expression = x + 39
>>> expression == another_expression
True
>>> float(expression)
139.0
>>> float(another_expression)
139.0
This is pretty much analogous to a tuple of mutable objects. The tuple itself may never change, but its contents may:
>>> a = [1, 2, 3]
>>> my_tuple = (a, 'something')
>>> my_tuple
([1, 2, 3], 'something')
>>> a[:] = ['changed'] # Only changing the contents of the list
>>> another_tuple = (a, 'something')
>>> my_tuple == another_tuple
True
>>> my_tuple
(['changed'], 'something')
>>> another_tuple
(['changed'], 'something')
Behind value
is evaluate()
¶
You might want to know what is happening behind the scenes when you ask for expression.value
or float(expression)
. In that case, check out the method evaluate()
.
Optimization problems¶
Creating a problem¶
We use Friendly Sam to formulate MILP problems. The optimization library could be extended to allow other types of problems, too, but this is what is supported today.
Now, let’s begin with a full example of an optimization problem.
>>> import friendlysam as fs
>>>
>>> # Create the problem
>>> x = fs.VariableCollection('x')
>>> prob = fs.Problem()
>>> prob.objective = fs.Maximize(x(1) + x(2))
>>> prob.add(8 * x(1) + 4 * x(2) <= 11)
>>> prob.add(2 * x(1) + 4 * x(2) <= 5)
>>>
>>> # Get a solver and solve the problem
>>> solver = fs.get_solver()
>>> solution = solver.solve(prob)
>>> type(solution)
<class 'dict'>
>>> solution[x(1)]
1.0
>>> solution[x(2)]
0.75
The solver does not in any way affect the problem or the variables. It just reads the problem, solves it and handles back a dict
with your Variable objects as keys and their solutions as values.
If you set the value
of some variables, those will be inserted into the problem before solving it:
>>> x(1).value = 0
>>> solution = solver.solve(prob)
>>> solution
{<friendlysam.opt.Variable at 0x...: x(2)>: 1.25}
>>> x(1) in solution
False
x(1)
is not in the solution, because you already set its value, so it was handled like a number by the solver.
Debugging constraints¶
Now let’s add another constraint:
>>> x(1).value = 0
>>> prob.add(1 <= x(1))
>>> solver.solve(prob)
Traceback (most recent call last):
...
friendlysam.opt.ConstraintError: The expression in <Constraint: Ad hoc constraint> evaluates to False, so the problem is infeasible.
In this case it’s obvious why the problem could not be solved. But for argument’s sake, let’s say we didn’t know which constraint was causing a problem. The error message was not too helpful, but the ConstraintError
luckily also contains a reference to the constraint that failed, so we can pick it out like this:
>>> try:
... solver.solve(prob)
... except fs.ConstraintError as e:
... failed_constraint = e.constraint
... print(repr(failed_constraint))
... print(repr(failed_constraint.expr))
... print(failed_constraint.expr)
... print(failed_constraint.desc)
... print(failed_constraint.origin)
...
<friendlysam.opt.Constraint at 0x...>
<friendlysam.opt.LessEqual at 0x...>
1 <= x(1)
Ad hoc constraint
None
OK, that’s helpful! We got the problematic constraint out. And there are a few things you should note.
- The type of the failed constraint is
friendlysam.opt.Constraint
. It was automatically created when we added afriendlysam.opt.LessEqual
constraint to the problem, and its sole purpose is to wrap the inequality1 <= x(1)
and to add some metadata.- The
Constraint
object contains theLessEqual
object that we added to the problem.- The
Constraint
object contains also a descriptiondesc
and a variable calledorigin
which is supposed to say something about where the constraint comes from.
Note
There is a quicker way of printing out some info about a constraint: long_description
:
>>> print(failed_constraint.long_description)
<friendlysam.opt.Constraint at 0x...>
Description: Ad hoc constraint
Origin: None
If you want to make your model easier to debug, you can use Constraint
instances with custom description and/or origin, like in this stupid example:
>>> from friendlysam import Constraint
>>> def constr(var, parameter):
... return var / 42 >= parameter
>>> for i in range(5):
... expr = constr(x(i), i)
... origin = (constr, x(i), i)
... prob += Constraint(expr, desc='Some description', origin=origin)
...
Different ways to add constraints¶
Note
In the examples above, we added constraints like this:
>>> prob.add(8 * x(1) + 4 * x(2) <= 11)
>>> prob += Constraint(expr, desc='Some description', origin=origin)
These two methods are equivalent, so just choose the syntax you like best.
You can also send an iterable (even a generator), and the items in the iterable can also be iterables, e.g:
>>> prob += ([constr(x(i), i), constr(x(i+1), i)] for i in range(5))
See the documentation for add()
for all the details.
Model basics: Parts and constraints¶
Interconnected parts¶
Friendly Sam is made for optimization-based modeling of interconnected parts, producing and consuming various resources. For example, an urban energy system may have grids for district heating, electric power, fossil gas, etc. Consumers, producers, storages and other parts are connected to each other through these grids. To describe these relations, Friendly Sam models make heavy use of the Part
class and its subclasses like Node
, FlowNetwork
, Cluster
, Storage
, etc. We will introduce these in due time, but first a few general things about Part
.
Parts have indexed constraints¶
A Part
typically represents something physical, like a heat consumer or a power grid. You can attach constraint functions to parts. Constraint functions are probably easiest to explain with a concrete example:
>>> from friendlysam import Part, namespace, VariableCollection
>>> class ChocolateFactory(Part):
... def __init__(self):
... with namespace(self):
... self.output = VariableCollection('output')
... self.constraints += self.more_than_last
...
... def more_than_last(self, time):
... last = self.step_time(time, -1)
... return self.output(last) <= self.output(time)
...
OK, what happens above is the following: We define ChocolateFactory
as a subclass of Part
. Upon setup, in __init__()
, we add a constraint function called more_than_last
, which defines the (admittedly bizarre) rule that the factory may never decrease its output from one time step to the next.
In Friendly Sam’s vocabulary, the time
argument in the example above is called an index. Our typical use case for indexing is a discrete time model, where each hour, day, year, or whatever time period, is an index of the model, and each constraint “belongs” to a time step just like in the silly example above.
Going back to the example, we can get the constraints out by making a ChocolateFactory
instance and calling constraints.make()
with an index:
>>> chocolate_factory = ChocolateFactory() # Create an instance
>>> constraints = chocolate_factory.constraints.make(47)
>>> constraints
{<friendlysam.opt.Constraint at 0x...>}
>>> for c in constraints:
... print(c.expr)
...
ChocolateFactory0001.output(46) <= ChocolateFactory0001.output(47)
The result of constraints.make(47)
is a set with one single constraint in it, saying that output at “time” 47 must be greater than or equal to output at “time” 46.
Note
In the example above, we wrote last = self.step_time(time, -1)
instead of just last = t-1
. This is because Part
has a bunch of nice functions to help out with time indexing. Read the API documentation for step_time()
. Also check out times()
and times_between()
. For example, because we used step_time(...)
, we can easily change the time representation of our chocolate factory like this:
>>> from pandas import Timestamp, Timedelta
>>> chocolate_factory.time_unit = Timedelta('6h')
>>> constraints = chocolate_factory.constraints.make(Timestamp('2015-06-10 18:00'))
>>> for c in constraints:
... print(c.expr)
...
ChocolateFactory0001.output(2015-06-10 12:00:00) <= ChocolateFactory0001.output(2015-06-10 18:00:00)
Advanced indexing¶
Indexing is really just a way to organize constraints in groups that belong together. When we have a whole bunch of parts, to get all the constraints that belong together, we can do things like this:
>>> from itertools import chain
>>> parts = Part(), Part(), Part() # Put something more useful here...
>>> some_index = 'could be anything'
>>> constraints = set.union(*(p.constraints.make(some_index) for p in parts))
Indexing is typically used to represent time, but it is really up to you to decide what an index means, and what to use as indices. We call it “index” rather than “time” because it is something more general than a representation of time. In fact, any hashable object can be used as an index, so you can do all sorts of complicated things. Examples of indexing can be found in the docs for step_time()
. Also check out times()
and times_between()
.
Friendly Sam currently has no mechanism for using constraint functions without indices. If you want to make a static model and really don’t need indexing, then just use some common index like None
or 0
for everything. (Or come up with a better solution and discuss it with us on GitHub.)
In the next section Flow networks: Nodes and resources you will also see how indexing is used to represent time in flow networks.
Flow networks: Nodes and resources¶
Note
This tutorial does not cover everything. To learn more, follow the links into the API reference for Node
, FlowNetwork
, Cluster
etc.
Friendly Sam makes it easy to formulate optimization problems with flow networks. Let’s begin with an example.
Nodes and balance constraints¶
An example¶
Custom types of nodes should typically be created by subclassing Node
, like this:
>>> from friendlysam import Node, VariableCollection, namespace
>>> class PowerPlant(Node):
... def __init__(self):
... with namespace(self):
... x = VariableCollection('output')
... self.production['power'] = x
...
>>> class Consumer(Node):
... def __init__(self, demand):
... self.consumption['power'] = lambda time: demand[time]
...
We have now defined a PowerPlant
class inheriting Node
, and a Consumer
class, also inheriting Node
. The power plant has its production['power']
equal to a VariableCollection
, and the consumer has consumption['power']
equal to the value found in the argument demand
. Let’s create instances and test them:
>>> power_plant = PowerPlant()
>>> power_plant.production['power'](3)
<friendlysam.opt.Variable at 0x...: PowerPlant0001.output(3)>
>>> power_demand = [25, 30, 33, 29, 27]
>>> consumer = Consumer(power_demand)
>>> consumer.consumption['power'](3)
29
Now connect the two nodes:
>>> from friendlysam import FlowNetwork
>>> power_grid = FlowNetwork('power', name='Power grid')
>>> power_grid.connect(power_plant, consumer)
>>> power_grid.children == {power_plant, consumer}
True
The Consumer
instance and the PowerPlant
instance were added to the power grid, and can now be found as children
of the FlowNetwork
.
Note
In this example, we use the key 'power'
in a few different places. Whatever we put as a key in a production
or consumption
dictionary, or a similar place, is called a resource. You are not limited to strings like 'power'
but could use any hashable type: numbers, tuples, most other objects, etc.
Balance constraints¶
Each Node
has a pre-defined constraint function for balance constraints, so calling constraints.make()
on the nodes creates balance constraints. The dictionaries production
and consumption
are automatically included in these balance constraints. The connect()
call creates a flow between two nodes, and it adds this flow to the appropriate outflows
or inflows
on those two nodes. Each Node
can then formulate its own balance constraints:
>>> for part in [consumer, power_plant, power_grid]:
... for constraint in part.constraints.make(3):
... print(constraint.long_description)
... print(constraint.expr)
... print()
...
<friendlysam.opt.Constraint at 0x...>
Description: Balance constraint (resource=power)
Origin: CallTo(func=<bound method Consumer.balance_constraints of <Consumer at 0x...: Consumer0001>>, index=3, owner=<Consumer at 0x...: Consumer0001>)
Power grid.flow(PowerPlant0001-->Consumer0001)(3) == 29
<friendlysam.opt.Constraint at 0x...>
Description: Balance constraint (resource=power)
Origin: CallTo(func=<bound method PowerPlant.balance_constraints of ...>, index=3, owner=<PowerPlant at 0x...: PowerPlant0001>)
PowerPlant0001.output(3) == Power grid.flow(PowerPlant0001-->Consumer0001)(3)
How balance constraints are made¶
Here are a few simple rules for how balance constraints are made:
Each
Node
has the five dictionariesconsumption
,production
,accumulation
,inflows
, andoutflows
.Whatever you decide to put as a key in any of these dictionaries is called a resource.
For each resource present in any of the dictionaries, the
Node
produces balance constraints like this:(sum of inflows) + production = consumption + accumulation + (sum of outflows)
The constraints of the node are accessed by calling something like
>>> index = 3 >>> constraints = power_plant.constraints.make(index)The index is passed on to the functions:
production[resource](index)
,consumption[resource](index)
, etc. In this way, indices always represent time when you are working with nodes and flow networks. You can use any function or object asproduction[resource]
,consumption[resource]
, etc, as long as it is callable.
Note
A Node
instance will always produce balance constraints for each of its resources
. Let’s say we had not connected the PowerPlant
instance to the consumer, then its balance constraint would be PowerPlant0001.output(3) == 0
. (Try it yourself!) In other words, flows of resources must always be balanced in a Friendly Sam model. Noone may produce a resource like 'power'
if it has nowhere to go, and noone can consume it unless there is a source.
Custom names¶
Note
You can name your Node
instances if you want something more personal than PowerPlant0001
. Just set the property name
, for example in the __init__
function, like this:
>>> class CHPPlant(Node):
... def __init__(self, name=None):
... if name:
... self.name = name
... ...
>>> chp_plant = CHPPlant(name='Rya KVV')
>>> chp_plant.name == str(chp_plant) == 'Rya KVV'
True
FlowNetwork¶
A FlowNetwork
essentially does two things: It creates the variable collections representing flows in the network, and it modifies the inflows
and outflows
of nodes when you call connect()
.
Unidirectional by default¶
Connections are unidirectional, so when you connect(node1, node2)
things can flow from node1
to node2
. Make the opposite connection if you want a bidirectional flow, or use this shorthand:
>>> power_grid.connect(power_plant, consumer, bidirectional=True)
Flow restrictions¶
To limit the flow between two nodes, get the flow VariableCollection
and set its upper bound ub
:
>>> flow = power_grid.get_flow(power_plant, consumer)
>>> flow
<friendlysam.opt.VariableCollection at 0x...: Power grid.flow(PowerPlant0001-->Consumer0001)>
>>> flow.ub = 40
Clusters and multi-area models¶
A cluster is fully connected¶
Sometimes we are not interested in making a full network model specifying all the flows between different nodes. The Cluster
class is a handy type of Node
for that. It is a type of node that can contain other nodes, and it essentially acts like a fully connected network, where all nodes are connected to all others.
When a Node
is put in a Cluster
, the child Node
will no longer make balance constraints, and instead the Cluster
creates an aggregated balance constraint, summing up the production
, consumption
and accumulation
of its contained children
.
>>> from friendlysam import Cluster
>>> power_plant = PowerPlant()
>>> consumer = Consumer(power_demand)
>>> power_cluster = Cluster(power_plant, consumer, resource='power', name='Power cluster')
>>> for part in power_cluster.descendants_and_self:
... for constraint in part.constraints.make(2):
... print(constraint.long_description)
... print(constraint.expr)
...
<friendlysam.opt.Constraint at 0x...>
Description: Balance constraint (resource=power)
Origin: CallTo(func=<bound method Cluster.balance_constraints ...>, index=2, owner=<Cluster at 0x...: Power cluster>)
PowerPlant0002.output(2) == 33
Multi-area models¶
A Cluster
instance can be used like any other Node
, for example in a FlowNetwork
. This is a simple way of making a multi-area model of, say, a district heating system. Let’s say the system has a few areas with significant flow restrictions between them. Then create a flow network with interconnected clusters, something like this:
area_A = Cluster(*nodes_in_area_A, resource='heat')
area_B = Cluster(*nodes_in_area_B, resource='heat')
area_C = Cluster(*nodes_in_area_C, resource='heat')
heat_grid = FlowNetwork('heat')
heat_grid.connect(area_A, area_B, bidirectional=True, capacity=ab)
heat_grid.connect(area_A, area_C, bidirectional=True, capacity=ac)
heat_grid.connect(area_B, area_C, bidirectional=True, capacity=bc)
Time in flow networks¶
It is natural to think of indices like time periods: All the expressions for flows, production and consumption must add up, for each index (time period). As shown in the examples above, the balance constraints for an index is called by passing the index to production
, consumption
, outflows
and inflows
.
There is another dictionary which is always used in balance constraints: accumulation
. It works just like the dictionaries production
and consumption
. To learn more, read the API docs for Storage
, and look at this example:
>>> from friendlysam import Storage
>>> from pandas import Timestamp, Timedelta
>>> battery = Storage('power', name='Battery')
>>> battery.time_unit = Timedelta('3h')
>>> t = Timestamp('2015-06-10 18:00')
>>> print(battery.accumulation['power'](t))
Battery.volume(2015-06-10 21:00:00) - Battery.volume(2015-06-10 18:00:00)
Example model¶
To get a feeling for what’s possible with Friendly Sam, have a look at this example model:
API reference¶
Variables¶
Variable ([name, lb, ub, domain]) |
A variable to build expressions with. |
VariableCollection ([name]) |
A lazy collection of Variable instances. |
Domain |
Domain of a variable. |
namespace (name) |
Prefix variable names. |
Expressions¶
Operation |
An operation on some arguments. |
Add |
Addition operator. |
Sub |
Subtraction operator. |
Mul |
Subtraction operator. |
Sum |
A sum of items. |
dot (a, b) |
Make expression for the scalar product of two vectors. |
Relation |
Base class for binary relations. |
Less |
The relation “less than”. |
LessEqual |
The relation “less than or equal to”. |
Eq |
The relation “equals”. |
Constraints and optimization¶
get_solver ([engine, options]) |
Get a solver object. |
Problem () |
An optimization problem. |
Maximize (expr) |
A maximization objective. |
Minimize (expr) |
A minimization objective. |
Constraint (expr[, desc, origin]) |
An equality or inequality constraint. |
SOS1 (variables, **kwargs) |
Special ordered set, type 1 |
SOS2 (variables, **kwargs) |
Special ordered set, type 2 |
piecewise_affine (points[, name]) |
Create a piecewise affine expression and constraints. |
piecewise_affine_constraints (variables[, ...]) |
Constrains for a piecewise affine expression. |
Models¶
Part ([name]) |
A part of a model. |
Node ([name]) |
A node with balance constraints. |
FlowNetwork (resource[, name]) |
Manages flows between nodes. |
Cluster (*parts[, resource, name]) |
A node containing other nodes, fully connected. |
Storage (resource[, capacity, maxchange, name]) |
Simple storage model. |
ConstraintCollection (owner) |
Generates constraints from functions. |
MyopicDispatchModel ([t0, horizon, step, ...]) |
docstring for MyopicDispatchModel |
Utilities¶
get_list (func, indices) |
Get a list of function values at indices. |
get_series (func, indices, **kwargs) |
Get a pandas Series of function values at indices. |
Exceptions¶
ConstraintError (*args[, constraint]) |
Raised when there is something wrong with a Constraint . |
NoValueError |
Raised when a variable or expression has no value. |
SolverError |
A generic exception raised by a solver instance. |
InsanityError |
Raised when a sanity check fails. |