Simplify docker image #12

aruneko · 2019-07-08T08:01:11Z

I modified the docker images to simplify it.

For example, the image size of v3.0.0-dev-postgresql_11 decrease from 1.4GB to 920MB.

The changes is below.

Replace base image pgrouting/postgres with mdillon/postgis
- mdillon/postgis is based on official Postgres image, well-maintained, and built automatically.
- So we can use latest version of postgresql and postgis easily.
- And we can refer to an official postgresql docker image document.
- This is why, I removed postgres directory.
Remove osm2pgrouting from this docker image
- I think the responsibility of this docker image is only providing pgrouting.
- If you use osm2pgrouting to import OSM data, it is better to create a other container which includes osm2pgrouting.

In addition, I think it is required a CI for building 3.0.0-dev automatically (#4).
But to detect push for the development branch, we have to place CI settings in pgRouting repository.
So if we resolve this issue, we have to consider it.

cayetanobv · 2019-07-08T11:24:26Z

Thanks @aruneko . You have done in this PR a lot of work to improve the project!
I've been trying to simplify the way we prepare the base image because it generates a lot of work and it is difficult to maintain. This could be a good opportunity to improve that.

Several things I would like to discuss before proceed:

mdillon/postgis is a well mantained postgis base image but it has things I don't like:
- init file creates a specific use-case template (this init file shouldn't be in the image base): https://github.com/appropriate/docker-postgis/blob/master/11-2.5/initdb-postgis.sh
- same with update file: https://github.com/appropriate/docker-postgis/blob/master/11-2.5/update-postgis.sh
- The only interesting part of Dockerfile is postgis installation so we could change base of our image by postgres official and add apt install of postgis (version as ENV in Dockerfile). Postgis installation from apt is fast.
osm2pgrouting: It is true that not all people are using it but I think this is a useful tool (and little in terms of disk). When you install PostGIS several tools are installed by default (shp2pgsql and pgsql2shp) and not all people use it.
Regarding automatic build of development branch: I agree with you. Maybe this is the point to discuss again issue Automatic builds for 3.0.0-dev #4

@cvvergara and @dkastl: What do you think about it?

cayetanobv · 2019-07-08T11:25:48Z

2.6.1/pg10/Dockerfile

@@ -1,16 +1,46 @@
-FROM pgrouting/postgres:pg10
+FROM mdillon/postgis:10


Discussing this change in main thread: #12 (comment)

aruneko · 2019-07-09T06:35:41Z

Thanks for your opinion!

I understand your concern, and I propose this Dockerfile.

for pgrouting 2.6.2

FROM postgres:11

ENV POSTGIS_MAJOR 2.5
ENV PGROUTING_VERSION 2.6.2

RUN apt update \
 && apt install -y \
        build-essential \
        cmake \
        perl \
        wget \
        libboost-graph-dev \
        libcgal-dev \
        libpq-dev \
        # install postgis from apt
        postgresql-${PG_MAJOR}-postgis-${POSTGIS_MAJOR} \
        postgresql-server-dev-${PG_MAJOR} \
 && cd /usr/local/src \
 && wget https://github.com/pgRouting/pgrouting/archive/v${PGROUTING_VERSION}.tar.gz \
 && tar xvf v${PGROUTING_VERSION}.tar.gz \
 && cd pgrouting-${PGROUTING_VERSION} \
 && mkdir build \
 && cd build \
 && cmake .. \
 && make \
 && make install \
 && cd ../../ \
 && rm -rf ./* \
 && apt purge -y \
        build-essential \
        cmake \
        perl \
        wget \
        libcgal-dev \
        libpq-dev \
        postgresql-server-dev-${PG_MAJOR} \
 && apt install -y \
        postgresql-${PG_MAJOR}-postgis-${POSTGIS_MAJOR}-scripts \
        libboost-atomic1.62.0 \
        libboost-chrono1.62.0 \
        libboost-graph1.62.0 \
        libboost-date-time1.62.0 \
        libboost-program-options1.62.0 \
        libboost-system1.62.0 \
        libboost-thread1.62.0 \
        libcgal12 \
 && apt autoremove -y \
 && rm -rf /var/lib/apt/lists/*

for pgrouting 3.0.0

FROM postgres:11

ENV POSTGIS_MAJOR 2.5
ENV PGROUTING_VERSION develop

RUN apt update \
 && apt install -y \
        build-essential \
        cmake \
        perl \
        wget \
        libboost-graph-dev \
        libpq-dev \
        # install postgis from apt
        postgresql-${PG_MAJOR}-postgis-${POSTGIS_MAJOR} \
        postgresql-server-dev-${PG_MAJOR} \
 && cd /usr/local/src \
 && wget https://github.com/pgRouting/pgrouting/archive/${PGROUTING_VERSION}.tar.gz \
 && tar xvf ${PGROUTING_VERSION}.tar.gz \
 && cd pgrouting-${PGROUTING_VERSION} \
 && mkdir build \
 && cd build \
 && cmake .. \
 && make \
 && make install \
 && cd ../../ \
 && rm -rf ./* \
 && apt purge -y \
        build-essential \
        cmake \
        perl \
        wget \
        libpq-dev \
        postgresql-server-dev-${PG_MAJOR} \
 && apt install -y \
        postgresql-${PG_MAJOR}-postgis-${POSTGIS_MAJOR}-scripts \
        libboost-graph1.62.0 \
        libboost-program-options1.62.0 \
 && apt autoremove -y \
 && rm -rf /var/lib/apt/lists/*

regarding osm2pgrouting

The base image postgres:11 is based on Debian stretch. The repository of Debian stretch includes an osm2pgrouting package but the version is 2.2.0, so we cannot install latest version of osm2pgrouting and have to build it for installing. I concern about to be more complex Dockerfile.

I understand the needs of osm2pgrouting, but for production deployment using public cloud (e.g. AWS, GCP), I hope to save as more disk space as possible. So, I think it is required both of docker image which includes it or not. Because, once we create a docker image, we cannot remove any files from it.

But I concern about increase of tags. If possible, I hope to remove support of 2.6.1.

dkastl · 2019-07-09T06:43:37Z

Would it be too much work or a bad idea to make a different Docker image for osm2pgrouting?

I think, that it's good to have a way to install the latest one as a Docker container, because distributions are sometimes slow to update. And I understand, that some people won't need it.

Not sure, what is the most reasonable solution. But I'm very glad about the discussion and the pull request of @aruneko . It actually shows users of this Docker image, that it is well thought and maintained.

cayetanobv · 2019-07-09T16:14:30Z

@aruneko I like new Dockerfiles you are proposing. Could you push these changes to PR?

Osm2pgrouting issue

I understand your concern about image size and I think in production is important.
@dkastl It looks good to me to maintain 2 Dockerfiles (one of them including osm2pgrouting) by pRouting version (we can automate it). The question here is: How we could tag osm2pgrouting image in a clearly way? What do you think about it?

woodbri · 2019-07-09T19:16:02Z

On 7/9/2019 12:14 PM, Cayetano Benavent wrote: @aruneko <https://github.com/aruneko> I like new Dockerfiles you are proposing. Could you push these changes to PR? Osm2pgrouting issue I understand your concern about image size and I think in production is important. @dkastl <https://github.com/dkastl> It looks good to me to maintain 2 Dockerfiles by pRouting version (we can automate it). The question here is: How we could tag osm2pgrouting image in a clearly way? What do you think about it? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#12?email_source=notifications&email_token=AAGETAUPC7ELROZ4HWHA3SLP6S2OPA5CNFSM4H6Y5QKKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZQYQRQ#issuecomment-509708358>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAGETAV7ZPMCGNBZLYQMFE3P6S2OPANCNFSM4H6Y5QKA>.

How about calling it something like pgRouting-plus? or -fat? to imply that there are additional things in it. Today it would have osm2pgrouting, but later we might want to add other stuff to it.

…

--- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus

aruneko · 2019-07-10T08:39:55Z

I pushed suggested Dockerfiles.

Then, I propose Dockerfile for a docker image includes osm2pgrouting.

FROM pgrouting/prgouring:3.0.0-dev-11

ENV OSM2PGROUTING_VERSION 2.3.6

RUN apt update \
 && apt install -y \
        build-essential \
        cmake \
        wget \
        libboost-program-options-dev \
        libexpat1 \
        libexpat-dev \
        libosmium2-dev \
        libpqxx-dev \
        zlib1g-dev \
 && cd /usr/local/src \
 && wget https://github.com/pgRouting/osm2pgrouting/archive/v${OSM2PGROUTING_VERSION}.tar.gz \
 && tar xvf v${OSM2PGROUTING_VERSION}.tar.gz \
 && cd osm2pgrouting-${OSM2PGROUTING_VERSION} \
 && mkdir build \
 && cd build \
 && cmake .. \
 && make \
 && make install \
 && cd ../tools/osmium/ \
 && mkdir build \
 && cd build \
 && cmake .. \
 && make \
 && make install \
 && cd /usr/local/src \
 && rm -rf ./* \
 && apt purge -y \
        build-essential \
        cmake \
        wget \
        libexpat-dev \
        libosmium2-dev \
        libpqxx-dev \
        zlib1g-dev \
 && apt install -y \
        libpqxx-4.0v5 \
 && apt autoremove -y \
 && rm -rf /var/lib/apt/lists/*

cayetanobv · 2019-07-11T10:20:56Z

Thanks @aruneko . Dockerfile for osm2pgrouting looks good to me (only a typo in name: pgrouting/prgouring:3.0.0-dev-11 -> pgrouting/pgrouting:3.0.0-dev-11). Could you push these changes.

Naming

I think we only need to find a name for this image to differentiate from "main". Following the proposal by @woodbri , I've added another name more (added to docker image "tag"):

-plus
-fat
-ext (stands for extended version)

What do you think?

dkastl · 2019-07-11T14:42:46Z

-plus sounds as it would be better.
What about -extra or -osm?

cayetanobv · 2019-07-11T15:19:39Z

-plus sounds as it would be better.
What about -extra or -osm?

@dkastl I like -extra. I think the goal is very clear.

cayetanobv · 2019-07-11T15:26:53Z

3.0.0-dev/pg10/README.md

@@ -1,3 +1,3 @@
 # pgRouting 3.0.0-dev (pg10)

-pgRouting Docker image (version 3.0.0-dev) built over [Postgres 10 with PostGIS](../../postgres/pg10) and dependencies.
+pgRouting Docker image (version 3.0.0-dev) built over [Postgres 10 with PostGIS](https://hub.docker.com/r/mdillon/postgis) and dependencies.


Change link url.

woodbri · 2019-07-11T15:29:26Z

+1 on -extra. better to keep it generic then we can put other stuff into it also and not have to have a separate one for everything we might want to add in the future.

aruneko · 2019-07-13T01:37:44Z

I pushed Dockerfile for -extra, and fixed URL on README.

cayetanobv · 2019-07-15T11:53:44Z

Thank you @aruneko I'm going to review all changes to proceed with this PR.

cayetanobv · 2019-07-18T15:36:12Z

@aruneko I've changed base branch to development. I will merge with master after some testing.

aruneko · 2019-07-19T02:01:49Z

Thanks a lot!

aruneko added 14 commits July 8, 2019 14:07

simplify Docker image

82ad005

remove unused resource files

3753d20

simplify Docker image

e7840a2

remove unused resource files

74ee55c

fix download file name

95b1228

simplify docker image

de3cecf

remove unused resource files

d8ffa75

simplify docker image

784fe91

remove unused resource files

2283b3c

simplify docker image

434cf27

remove unused resource files

b6c8b40

simplify docker image

f1a9c70

remove unused resource files

f3fa1ff

remove unused postgres docker image

ed113a9

cayetanobv self-requested a review July 8, 2019 11:24

cayetanobv requested changes Jul 8, 2019

View reviewed changes

based on the original postgresql docker image

276fa39

cayetanobv reviewed Jul 11, 2019

View reviewed changes

aruneko added 2 commits July 13, 2019 10:32

support osm2pgrouting

2dc2095

fix URLs on README

fd74b6f

cayetanobv changed the base branch from master to development July 18, 2019 15:34

cayetanobv approved these changes Jul 18, 2019

View reviewed changes

cayetanobv merged commit ee375ea into pgRouting:development Jul 18, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Simplify docker image #12

Simplify docker image #12

aruneko commented Jul 8, 2019

cayetanobv commented Jul 8, 2019

cayetanobv Jul 8, 2019

aruneko commented Jul 9, 2019

dkastl commented Jul 9, 2019

cayetanobv commented Jul 9, 2019 •

edited

Loading

woodbri commented Jul 9, 2019 via email

aruneko commented Jul 10, 2019

cayetanobv commented Jul 11, 2019

dkastl commented Jul 11, 2019

cayetanobv commented Jul 11, 2019

cayetanobv Jul 11, 2019

woodbri commented Jul 11, 2019

aruneko commented Jul 13, 2019

cayetanobv commented Jul 15, 2019

cayetanobv commented Jul 18, 2019

aruneko commented Jul 19, 2019

		@@ -1,16 +1,46 @@
		FROM pgrouting/postgres:pg10
		FROM mdillon/postgis:10

Simplify docker image #12

Simplify docker image #12

Conversation

aruneko commented Jul 8, 2019

cayetanobv commented Jul 8, 2019

cayetanobv Jul 8, 2019

Choose a reason for hiding this comment

aruneko commented Jul 9, 2019

for pgrouting 2.6.2

for pgrouting 3.0.0

regarding osm2pgrouting

dkastl commented Jul 9, 2019

cayetanobv commented Jul 9, 2019 • edited Loading

Osm2pgrouting issue

woodbri commented Jul 9, 2019 via email

aruneko commented Jul 10, 2019

cayetanobv commented Jul 11, 2019

Naming

dkastl commented Jul 11, 2019

cayetanobv commented Jul 11, 2019

cayetanobv Jul 11, 2019

Choose a reason for hiding this comment

woodbri commented Jul 11, 2019

aruneko commented Jul 13, 2019

cayetanobv commented Jul 15, 2019

cayetanobv commented Jul 18, 2019

aruneko commented Jul 19, 2019

cayetanobv commented Jul 9, 2019 •

edited

Loading