README.md 12.7 KB
Newer Older
Justin Andresen's avatar
Justin Andresen committed
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
# Comparision of Haskell Effect Libraries

## Libraries

On Hackage, there are a lot of different effect libraries for Haskell.
The following is an alphabetically sorted list of packages that I've considered.

 - [`effect-handlers`][]
 - [`effect-monad`][]
 - [`effects`][]
 - [`effin`][]
 - [`extensible`][]
 - [`extensible-effects`][]
 - [`freer`][]
 - [`freer-effects`][]
 - [`freer-simple`][]
 - [`fused-effects`][]
 - [`polysemy`][]
 - [`simple-effects`][]

There are even more libraries such as [`eff`][] for example.
I've not considered [`eff`][] in my analysis since it is still under construction and hasn't been published to Hackage yet.

It would be infeasible to compare all of these libraries with each other.
Thus, I'll employ the *wisdom of the crowd* to determine which library it is worth to take a closer look at.
For this purpose, I've transferred usage statistics from Hackage into a [Google Sheets document][data].
A copy of the data can be found in the `data.csv` file.
To put the data points into perspective, I've also recorded data for the monad transformer library [`mtl`][].

## Library Popularity

### Most Downloads

The first data point I've considered is the total number of downloads.
The number of downloads will give us a rough estimate of which libraries are used the most.
The majority of packages don't seem to be very widespread with **between 1,500 and 7,500 downloads**.

The top three packages by total number of downloads are as follows.
They account for 58% of all downloads.

 1. [`extensible-effects`][] with a total of **29,470 downloads**
 2. [`extensible`][]         with a total of **22,451 downloads**
 3. [`simple-effects`][]     with a total of **13,268 downloads**

For comparison, the [`mtl`][] library has **461,739 downloads** in total.

### Most Recent Downloads

The total number of downloads only tells us whether a package once has been popular (or available for a long time).
However, it does not tell us how popular the package is right now.
The current popularity of a package is better estimated by the number of downloads within the last 30 days (*recent downloads* hereinafter).
For example, while the [`extensible-effects`][] and [`simple-effects`][] libraries account for 38% of all downloads, they were recently downloaded just 48 and 36 times, respectively, which corresponds to less than 4% of all recent downloads.
This could be an indicator that the packages have been superseded by other libraries.

The following are the top three libraries by recent downloads.

 1. [`fused-effects`][] with **697 recent downloads**
 2. [`polysemy`][]      with **674 recent downloads**
 3. [`extensible`][]    with **546 recent downloads**

With a **total of 1,917 recent downloads**, they account for 81% of all recent downloads.
Combined they are thus more popular than [`mtl`][], which was **downloaded 1,535 times** in the last 30 days.
Current usage of the other libraries seems to be neglectable.

### Most Votes

Hackage provides a voting system to rate packages which could in theory provide more insights into the actual popularity of packages.
Unfortunately, this feature does not seem to be widely accepted throughout the Hackage community.
For example, the [`mtl`][] package received a measly **15 votes** despite its popularity.
Thus, we cannot expect any statistically relevant results from the rating.
In fact, most packages were not voted for at all.
However, what I found very interesting is that [`fused-effects`][] and [`polysemy`][] (which scored best in the *Most Recent Downloads* test) **received 7 and 6 votes**, respectively.
Considering that [`mtl`][] has been downloaded over 30 times more often than both packages combined, I think that it is remarkable that they almost received the same number of votes.
This suggests that interest in these two packages is high even though their actual usage is still low.

### Most Stargazers

Another voting mechanism with better acceptance is provided by GitHub's starring feature.
Since all packages link to their GitHub pages on their Hackage page, we can analyze how many people have starred the packages.

 1. [`polysemy`][]           with **713 stargazers**
 2. [`fused-effects`][]      with **441 stargazers**
 3. [`freer-simple`][]       with **181 stargazers**
 4. [`extensible-effects`][] with **160 stargazers**

All other libraries have **less than 100 stargazers**.
Interestingly, the `mtl` library has only **216 stargazers** which confirms our hypothesis from the analysis of Hackage votes that `polysemy` and `fused-effects` are extremely popular in the Haskell community right now.
An analysis of watchers and forks shows similar results.

It is surprising that [`simple-effects`][] received only **4 stars** considering its high number of total downloads.
The bad ranking is probably due to the usage of GitLab instead of GitHub as a hosting service.

### Summary

In summary, [`extensible-effects`][] seems to be the most widely used effect library at the moment.
However, interest in this library recently seems to have dropped in favor of [`fused-effects`][] and [`polysemy`][].
The [`extensible`][] library also seems to be used widely.
The high number of recent downloads suggests that it is still in use today.
[`freer-simple`][] and [`simple-effects`][] have medium popularity.

## Library Maintenance

In addition to popularity, I think that it is also important to evaluate how well maintained a package is.
We neither want to adopt a library, that has been abandoned by its authors nor a library whose API is going to change frequently.

### Most Recently Updated

The [`fused-effects`][], [`extensible`][], and [`polysemy`][] libraries have last been **updated this year**.
Whereas [`freer-simple`][], [`effects`][] and [`extensible-effects`][] have been **updated in 2019** for the last time.
All other packages are over two years old.
However, the [`mtl`][] library also has been **updated in February 2018**.
Thus, it may also be a sign that an API has stabilized if the package hasn't been updated for a long time.

### Most Recently Committed

Since the last update does not seem to tell us whether a package is still maintained, we have to look at the Git activity.
There is a commit on the default branch of the following repositories that is predated by the last commit to [`mtl`][] (i.e., commits after October 2019).

 - [`polysemy`][]           with last commit from **June 2020**.
 - [`fused-effects`][]      with last commit from **June 2020**.
 - [`freer-simple`][]       with last commit from **May 2020**.
 - [`extensible-effects`][] with last commit from **April 2020**.
 - [`extensible`][]         with last commit from **March 2020**.
 - [`simple-effects`][]     with last commit from **January 2020**.

### Summary

All of the libraries that performed best in the popularity analysis above are still actively maintained.
Except for [`polysemy`][] and [`fused-effects`][], library updates on Hackage are infrequent, though.

## Library Features and Compatibility

Finally, we have to find out whether the libraries provide the features that we need and whether they are compatible with our setup.
We will only consider the libraries that were identified as actively maintained in the test above.

### Compatibility

In our setup we are using GHC 8.6.5.
The effect library should be compatible with that version of GHC as well as all libraries that we are using already.
Thus, the libraries have been added as a dependency to the `.cabal` file.
Since the plan is to use the same effect library for the [Free Compiler][freec] and the Free Compiler has more dependencies than `haskell-src-transformations`, this test was conducted against the Free Compiler directly.
In order to avoid side effects, each library was tested in its own Docker container.
No incompatibilities were found.
I also didn't notice any significant differences in build time.

The scripts that I've used to perform the compatibility check can be found in the `compatibility` directory.

### Features Comparision

Most of the libraries seem to provide very similar features.
They mostly differ in the concrete syntax for effect sets, how new effects are declared and their internal representation of effects.

#### [`polysemy`][] and [`fused-effects`][]

The advantage of [`polysemy`][] and [`fused-effects`][] over other effect libraries seems to be their performance and support for higher-order effects.
Prior to GHC 8.8 the [`polysemy`][] is outperformed by [`fused-effects`][].
In GHC 8.9+ [`polysemy`][] can take advantage of compiler optimizations to archive "zero-cost".
Since we are using GHC 8.6.5, we would not be able to take advantage of [`polysemy`][]'s performance advantages.
However, [`polysemy`][] reportedly provides better type errors and defining new effects requires less boilerplate code.
One drawback of [`polysemy`][] is that it needs a lot more language extensions than [`fused-effects`][].
Additionally, [`polysemy`][] can use a compiler plugin to improve type inference.

#### [`freer-simple`][], [`extensible-effects`][] and [`simple-effects`][]

The other libraries can only model first-order effects.
Thus functions like `catchError` and `local` cannot be implemented as effects and must be given a fixed interpretation.
I still think that they are sufficient for our purposes.

[`freer-simple`][] and [`simple-effects`][] require a similar number of language extensions to [`fused-effects`][].
The language extensions required by [`extensible-effects`][] is not documented.

Similar to [`polysemy`][], [`freer-simple`][] also provides custom compile-time errors and has `TemplateHaskell` integration to reduce the amount of boilerplate code that has to be written to define new effects.

#### [`extensible`][]

Event though [`extensible`][] still claims to provide "extensible, efficient, optics-friendly data types and effects", the effects have been moved to [a separate package][`extensible-skeleton`] six months ago.
Unfortunately, the library is not documented very well.
I'll not consider it in my further analysis.

### Summary

[`polysemy`][] and [`fused-effects`][] sound interesting but we probably don't need higher-order effects and performance is not critical in our case.
However, improved compile-time error messages and better type inference of [`polysemy`][] could be useful for us.

## Test Projects

The `test-projects` directory contains Cabal projects for the different libraries for a direct comparison.
The test projects can be executed as follows.

```haskell
$> cd ./test-projects/<library>/
$> cabal new-repl
λ> :m MyProgs
λ> runMyProg
(42,True)
λ> logMyProg
Info: 84
Info: 42
(42,True)
```

[data]:
  https://docs.google.com/spreadsheets/d/1Zmy6E5ji8wGEKBWcYYUtj89r7a2pIrUCyGhCwiptxas/edit?usp=sharing
  "Effect Libraries — Google Sheets"
[freec]:
  https://github.com/FreeProving/free-compiler
  "Free Compiler on GitHub"

[`effect-handlers`]:
  https://hackage.haskell.org/package/effect-handlers
  "effect-handlers: A library for writing extensible algebraic effects and handlers. Similar to extensible-effects but with deep handlers."
[`effect-monad`]:
  https://hackage.haskell.org/package/effect-monad
  "effect-monad: Embeds effect systems and program logics into Haskell using graded monads and parameterized monads"
[`effects`]:
  https://hackage.haskell.org/package/effects
  "effects: Computational Effects"
[`eff`]:
  https://github.com/hasura/eff
  "eff — screaming fast extensible effects for less"
[`effin`]:
  https://hackage.haskell.org/package/effin
  "effin: A Typeable-free implementation of extensible effects"
[`extensible`]:
  https://hackage.haskell.org/package/extensible
  "extensible: Extensible, efficient, optics-friendly data types and effects"
[`extensible-effects`]:
  https://hackage.haskell.org/package/extensible-effects
  "extensible-effects: An Alternative to Monad Transformers"
[`extensible-skeleton`]:
  https://hackage.haskell.org/package/extensible-skeleton
  "extensible-skeleton: Operational-based extensible effect library"
[`freer`]:
  https://hackage.haskell.org/package/freer
  "freer: Implementation of the Freer Monad"
[`freer-effects`]:
  https://hackage.haskell.org/package/freer-effects
  "freer-effects: Implementation of effect system for Haskell."
[`freer-simple`]:
  https://hackage.haskell.org/package/freer-simple
  "freer-simple: Implementation of a friendly effect system for Haskell."
[`fused-effects`]:
  https://hackage.haskell.org/package/fused-effects
  "fused-effects: A fast, flexible, fused effect system."
[`mtl`]:
  https://hackage.haskell.org/package/mtl
  "mtl: Monad classes, using functional dependencies"
[`polysemy`]:
  https://hackage.haskell.org/package/polysemy
  "polysemy: Higher-order, low-boilerplate, zero-cost free monads."
[`simple-effects`]:
  https://hackage.haskell.org/package/simple-effects
  "simple-effects: A simple effect system that integrates with MTL"