9 Replies Latest reply: Aug 31, 2018 6:12 AM by Marcus Sommer RSS

    Extremely long time to load data from Postgres

    Gal Schkolnik

      Hi there,

       

      I'm totally new to QlikSense Desktop, so to try it out I've decided to add one of my smallest tables from my Postgres database, with 10M rows, with no joins or anything fancy at first.
      Data load has been in progress for the last 4 hours.

       

      Is this normal? Can it be I'm doing something wrong?

       

      Looking forward to finding out what I can expect from this tool!

       

      Bests,

       

      Gal.

        • Re: Extremely long time to load data from Postgres
          omar bensalem

          Hell no !

          Something surely went wrong; Qlik won't ever take that long for 10M rows ! It would take at most a few minutes (AT MOST)

          • Re: Extremely long time to load data from Postgres
            Derek Emlet

            Hi Gal,


            I have had QlikSense take over an hour to load 1M records from SalesForce using the auto generated script with no modifications. Depending on your data it can get hung up from what I have experienced. For example, I had a description field that that contains paragraphs of text, and QlikSense didn't seem to know what to do with it. The load would start and no text would appear and the timer would spin for hours. I commented out the field and it loaded normally.

             

            Check what data your loading from Postgres, you might need to comment out some columns/fields to see if anything could be causing the lag.

             

            Hope this helps

             

            -Derek

            • Re: Extremely long time to load data from Postgres
              Marcus Sommer

              Qlik has nothing to do with the query unless it transferred it to the database and received any output from there. If it's quite slow like in your case then it means that your database respectively this query is slow and/or the database-driver is not really suitable and/or the network to Qlik is slow.

               

              - Marcus

                • Re: Extremely long time to load data from Postgres
                  Gal Schkolnik

                  Hi Marcus,

                  Thanks for your response.

                  It isn't my DB, as loading the same table to Tableau Desktop only takes about 15 minutes.

                  It isn't the internet, as I am using QlikSense Desktop.

                  There is no query, I just tried to load the whole table, so it isn't the query either.

                  So are you saying that the QlikSense driver for Postgres is no good?

                    • Re: Extremely long time to load data from Postgres
                      Marcus Sommer

                      I'm not familar with loading data from Postgres but if it worked in Tableau in 15 minutes it should be no different within Sense. Are you sure that you query the same database - querying a productive database and dump of it could make a big difference. Also if there are maybe different priorities between the querying clients.

                       

                      - Marcus

                        • Re: Extremely long time to load data from Postgres
                          Gal Schkolnik

                          Hi Marcus,

                          I didn't want to say anything the first time, because I thought you just weren't aware that I had already checked these possibilities, but your responses are extremely disrespectful. I explained the situation to you as it is, no need to ask me if I know what database I'm querying

                          I know it's hard, but please try to assume other people also know what they're doing.

                          Bests,

                          Gal.

                            • Re: Extremely long time to load data from Postgres
                              Marcus Sommer

                              Hi Gal,

                               

                              sorry - if I made this impression to you. Many people which ask questions here within the community are quite unexperienced and don't check much possibilities or use the search before asking and quite often are the usual common hints enough to resolve a problem.

                               

                              Like above mentioned there should be no big difference between Tableau and Sense. This means there must be something different between both loadings which caused such a delay but I have not much ideas what it could be and I would probably apply the method of elimination to exclude one possible reason after another.

                               

                              Possible actions here could be to pull the data with both tools from the same machine with the same user multiple times with a various number of records - maybe starting with 10 k - and with complete empty applications (no other data, variables or any objects within the UI).

                               

                              Of course this a quite ugly trial & error approach but without a faint idea it's rather difficult to find the cause and therefore I hope it will be nevertheless helpful for you.

                               

                              - Marcus