Re: Testing: Fixtures vs Factories vs ???

I work on a complex django project -- our test suite is over 1000
tests. We've begun converting them from fixtures to factories. Here's
why:

- Factories were measurably faster. At least in my local environment,
which may be more IO bound than others, but I was looking at the
results of tests and seeing tests run up to ten times faster using
factories. For example, I'd run a class including 10 tests that used
three fixtures and a total of maybe 30 instances a few times, and get
an average run time of about say 30 seconds. Then I would convert the
same test to use data from factories instead, and the same class would
run in under 4 seconds. The same speed gain remained in our jenkins
environment. My results may not be typical.

- At least in some cases, data loaded in factories doesn't get
unloaded. I don't know if that was by design or not but it caught me
by surprise. So if you have a test that loads a fixture with three
users, and assert that there are exactly three users, this test will
fail if another test happens to load a fixture that has some other
user in it has already run. A nightmare to debug when a test in an
unrelated part of the test suite starts failing after an innocuous
change, as running this test by itself will pass every time. Whereas
data your test creates with a factory will be blown out before the
next test runs.

We still use fixtures in some cases: static data that lots of tests
use and none of them change.

Best,
- Steve

On Feb 17, 5:16 am, Paolo <paul...@gmail.com> wrote:
> Hi folks,
>
> I've been coding away on my latest Django project, and have recently
> started to think about how best to test the functionality I have so far.
>
> Creating fixtures for my tests seems like a good idea initially, but from
> experience I find fixtures become harder and harder to maintain as the
> project ages and expands.
>
> The literature appears to recommend fixtures as the way forward, but I also
> see there are projects such as 'factory boy' (inspired by the Rails world)
> offering an alternative (or complimentary?) method.
>
> Are there any recommendations/comments/examples from the Django community
> on what the best way to go is with regards to data for testing?
>
> Cheers,
>
> Paul

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.