Commit graph

1023 commits

Author SHA1 Message Date
Yuri Bochkarev
759fe78e62 Bump version (0.7.0 -> 0.8.0)
Features:
  - Add `--download-delay` option that adds a specified delay in seconds
    before downloading next course. This is useful when downloading many
    courses at once. Default value is 60 seconds.
  - Add `--only-syllabus` option which is when activated, allows to skip
    download of the course content. Only syllabus is parsed.
  - Add support for `reflect` and `mcqReflect` question types in quizzes.
  - Courses that encountered an error while parsing syllabus will be listed
    in the end of the program execution, after all courses have been
    processed (hopefully, downloaded). This helps skip vast output and easily
    see which courses need user's attention, e.g. enrollment, session
    switching or just patience until the course start date.

Bugfixes:
  - Locked programming assignments in syllabus used to crash coursera-dl.
    Now the script goes on parsing syllabus and skips locked assignments.
  - Add missing import statement to playlist generation module
2016-10-04 00:28:21 +03:00
Yuri Bochkarev
f21fd18b4f Update CHANGELOG.md 2016-10-04 00:26:50 +03:00
Yuri Bochkarev
f7d451c581 Fix tests to make sure they use mocked session, not actual Session
related #573
2016-10-03 20:48:03 +03:00
Yuri Bochkarev
00b36f7035 Show problematic courses after having downloaded all courses
This change makes the script show courses where an error
during syllabus parsing occurred. Printing such courses
in the end is helpful to bring attention when downloading
multiple courses.

related #573
2016-10-03 19:51:09 +03:00
Yuri Bochkarev
69de37b82e Add support for reflect and mcqReflect question type
reflect generates the same narrow edit field as for numerical answers.
Yet it does not look as on Coursera site, I think it's enough
to indicate the intention. mcqReflect is basically the same as just mcq.

related #573
2016-10-03 19:13:44 +03:00
Yuri Bochkarev
9f333903d0 Add --only-syllabus option to download syllabus only and skip course contents
Useful for developers.

related #573
2016-10-03 18:21:18 +03:00
Yuri Bochkarev
7579dc9771 Introduce --download-delay option
This option makes the script sleep for N seconds before
downloading next course. Default value is 60 seconds.

This option proved useful when downloading many courses at
once. Otherwise Coursera API starts to respond with errors
(cannot login).

related #573
2016-10-03 17:53:38 +03:00
Yuri Bochkarev
e430da1304 Be more verbose when printing login failure message
related #573
2016-10-03 17:37:39 +03:00
Yuri Bochkarev
7f773530fc Add missing import to playlist.py
related #573
2016-10-03 17:36:52 +03:00
Yuri Bochkarev
a1c1d624b4 Be more robust when extracting quiz/exam/programmingAssignment
This a revised version of #575 pull request by @partobs-mdp

fix #568
related #573
2016-10-03 17:36:17 +03:00
Yuri Bochkarev
4907634a9c Merge pull request #566 from CBickel87/master
Add Windows install & configuration to README.md
2016-08-17 18:00:37 +03:00
Charles Bickel
75e3726346 Update README.md 2016-08-17 09:16:07 -04:00
Charles Bickel
a8132fdb1c Update README.md 2016-08-17 09:04:56 -04:00
Charles Bickel
308fbc1857 Update README.md 2016-08-17 08:30:41 -04:00
Charles Bickel
584cbf3265 Update README.md 2016-08-17 08:26:15 -04:00
Charles Bickel
7b4c29cc2f Update README.md 2016-08-17 02:27:13 -04:00
Charles Bickel
84a8cb7b0f Update README.md 2016-08-16 16:10:13 -04:00
Yuri Bochkarev
ab9f4bfed3 Merge pull request #563 from andersonvom/master
Make class_names arg optional for certain flags
2016-08-08 14:22:20 +03:00
Yuri Bochkarev
1218a83926 Merge pull request #564 from nachikethas/docupdate
Fix command typo in alternative Unix installation
2016-08-08 14:21:33 +03:00
Anderson Mesquita
d3d1c4d0f1 Make class_names arg optional for certain flags
The `class_names` argument shouldn't be required for certain flags (e.g.
`--version` or `--list-courses`).  This makes it optional while also
taking these flags into account to display an appropriate error message
when they aren't being used.

Fixes #562
2016-08-06 10:09:38 -07:00
Nachikethas
053883eb23 Fix command typo in alternative Unix installation
By default, the git clone command clones the repository to a directory
with the repository's name.
2016-08-06 03:10:46 -07:00
Yuri Bochkarev
3cdc1f16a2 Bump version (0.6.1 -> 0.7.0)
Features:
  - Added option `--list-courses` to list currently enrolled courses (#514)
  - Added option `--jobs N` to download resources in N threads simultaneously (#553)
  - Added option `--download-quizzes` to download contents of graded and
    ungraded quizzes (#490)
  - Added option `--cache-syllabus` to avoid downloading course syllabus on
    every run (this option is rather for developers)

Bugfixes:
  - Locked lectures are also requested from server now, which allows to
    download some programming assignments that were not downloaded before (#555)

Deletions:
  - Support for old-style courses has been removed (Coursera discontinued old courses:
    http://coursera.tumblr.com/post/145882467032/courseras-transition-to-a-new-technology-platform)
  - `--ignore-http-errors` option has been removed and the default behavior
    has been adjusted to include this option
  - Removed deprecated `--on-demand` option. Now OnDemand classes are downloaded
    by default
2016-07-28 14:24:30 +03:00
Yuri Bochkarev
2dbe2fe0b6 Add link to coursera-dl aur package (ArchLinux) 2016-07-28 15:19:37 +04:00
Yuri Bochkarev
ad8ccf3f06 Update CHANGELOG.md 2016-07-28 14:10:35 +03:00
Yuri Bochkarev
bda29d9835 Fix error in --list-courses option 2016-07-28 12:56:44 +03:00
Yuri Bochkarev
7330014077 Merge pull request #559 from balta2ar/download-quizzes
Add support for quiz and exam download

To enable quiz/exam download, use `--download-quizzes`.

fix #490
related #546
2016-07-28 12:31:34 +03:00
Yuri Bochkarev
4f6ec33d51 Fix dict construction in python2.6 2016-07-28 11:58:19 +03:00
Yuri Bochkarev
ed8dce5472 Add fixtures for regex and mathExpression question types 2016-07-28 11:49:10 +03:00
Yuri Bochkarev
b53e176a7a Add basic support for regex and mathExpression question types
Unescape question and answer text.
2016-07-28 11:44:35 +03:00
Yuri Bochkarev
28fe48b488 Add --download-quizzes option
This option enables quiz and exam (graded and ungraded quizzes)
download.
2016-07-28 10:29:19 +03:00
Yuri Bochkarev
e19a533e9d Be more robust to errors during quiz download
If user has already depleted his number of attempts, report that
    and continue downloading other quizzes.# Please enter the commit message for your changes. Lines starting

HTTPError in debug run.

git di --

Be more verbose on cached
2016-07-27 23:10:03 +03:00
Yuri Bochkarev
716acf736a Remove several excessive methods from network.py 2016-07-27 22:30:57 +03:00
Yuri Bochkarev
9c4651cba4 Update/fix tests 2016-07-27 21:22:38 +03:00
Yuri Bochkarev
ca12539595 Add support for audio files in quizzes (they are embedded into the page)
Add <meta charset> tag so that Korean Hangul is displayed properly.
2016-07-27 14:46:26 +03:00
Yuri Bochkarev
5153cfddfd Fix bug when condition for url skipping was reversed 2016-07-27 14:17:21 +03:00
Yuri Bochkarev
2197e2bf80 Remove method _get_open_course_asset_urls 2016-07-26 23:29:14 +03:00
Yuri Bochkarev
865e789d76 Add image conversion test for MarkupToHTMLConverter
Use AssetRetriever in MarkupToHTMLConverter
Reduce AssetRetriever test to contain only unique typeNames
2016-07-26 21:38:42 +03:00
Yuri Bochkarev
a6228e9e22 Extend AssetRetriever test to include images 2016-07-26 20:43:22 +03:00
Yuri Bochkarev
787b3eb32e Add option download=True to AssetRetriever 2016-07-26 20:22:49 +03:00
Yuri Bochkarev
c31e0e0265 Add test for AssetRetriever 2016-07-26 20:07:10 +03:00
Yuri Bochkarev
ea0926e2e2 Add AssetRetriever helper class
This class should simplify asset retrieval.
2016-07-26 19:21:58 +03:00
Yuri Bochkarev
b7aace614e Add basic tests for MarkupToHTMLConverter 2016-07-25 12:53:53 +03:00
Yuri Bochkarev
5445143d20 Add tests for QuizExamToMarkupConverter 2016-07-25 00:24:58 +03:00
Yuri Bochkarev
f7a214fd1e Fix old tests 2016-07-24 22:56:54 +03:00
Yuri Bochkarev
6523ccdb17 Reorganize quiz/exam extraction methods 2016-07-24 22:33:45 +03:00
Yuri Bochkarev
6a77c95d77 Implement all question types in quiz->markup converter
Add CSS style for body, input to have decent margins.
2016-07-24 21:06:02 +03:00
Yuri Bochkarev
ebce6c8f45 WIP Draft support for quiz and exam downloading
I nearly went gray when I attacked this feature. One day
week ago I implemented quiz download and it worked fine.
I added HTML conversion, I could see quizzes offline. Then
I had no time to work on that for a week. One week later
I ran the script and it responded with 401 Unauthorized.
God, I don't know what happened in my code, maybe I fainted
and my head dropped on the keyboard and I accidentally
removed part of the code but it stopped working. Or maybe
Coursera servers were upgraded and their response changed.

It turned out that when we download quizzes/exams, we need
to send along all those additional cookies such as a pack of
csrf* cookies, csrf_token and (I spend hours debugging it!)
CAUTH token. In fact, if I had looked at cookies.py, I would
have found out it's already implemented for login method.

Anyway, for some reasons CAUTH is not sent in POST request
even if you use requests.Session. Thus I manually add those
cookies when I request quizzes.
2016-07-24 12:58:31 +03:00
Yuri Bochkarev
6be11a7244 WIP First attempt at implementing quiz download support
Nothing is working yet :(
2016-07-22 20:36:33 +03:00
Yuri Bochkarev
3835fd94bb Update CHANGELOG.md 2016-07-17 19:40:42 +03:00
Yuri Bochkarev
c205286a82 Improve conversion of instructions
* Use html.parser instead of html5lib in BeautifulSoup as
  the latter automatically tries to insert missing tags,
  but it only makes instructions look worse offline
  See http://stackoverflow.com/a/14822344/258421
* In course syllabus request from server locked items too
* Add css style for img tag to make sure that large
  images are scaled down to maximal page width, but
  small images remain the same
  See http://stackoverflow.com/a/18113104/258421
* Print lecture typeName during the parsing stage

fix #555
related #546
2016-07-17 19:31:09 +03:00