We are aware of the issue with the badge emails resending to everyone, we apologise for the inconvenience - learn more here.

Forum Discussion

fangzefunny's avatar
fangzefunny
Explorer | Level 3
3 years ago

dropbox.filesUpload sometimes does not work

Hi,

 

I am now doing an online experiment via URL and trying to collect data using dropbox javascript SDK.

At the end of the experiment, the success message was always prompted. But the data was not always uploaded to my dropbox. 

About 1/5 data was missing on the way to my dropbox.

 

I have no idea why the data is missing and need help with possible reasons.

Here is my code.  Each file is about 30kb, so the file size shouldn't be a problem.

 

I don't know if this is the right way to place the question. Is any extra information needed? 

 

 

 

 

require("dotenv").config();
var Dropbox = require("dropbox").Dropbox;
const fetch = require("node-fetch");

const dbx = new Dropbox({
    accessToken: #############,
    refreshToken: #############,
    clientId: #############, 
    clientSecret: #############,
    fetch
});

var saveDropbox = function(content, filename){
    dbx.filesUpload({
        path: "/" + filename,
        contents: content 
    })
};
app.post("/experiment-data/:filename", function(request, response){
    // convert json to csv 
    DATA_CSV = json2csv(request.body); 
    var filename = request.params.filename;
    savedropbox(DATA_CSV, filename);
    response.end();
});

function(){
   var sel_data = jsPsych.data.get();
   $.ajax({
     type: "POST",
     url:  "/experiment-data/" + filename,
     data: JSON.stringify(sel_data.values()),
     contentType: "application/json"
     }).done(function(){
          alert("You have compeleted the experiment and your data has been 
          saved! ");
    }).fail(function(){
           alert("A problem occurs while writing data to Dropbox. "
          + "Data will be saved to your computer. ";
          var csv = sel_data.csv();
          downloadCSV(csv, fname);  
    });

 

 

 

  


  • fangzefunny wrote:

    ...

    At the end of the experiment, the success message was always prompted. But the data was not always uploaded to my dropbox. 

    ...

    ...
    var saveDropbox = function(content, filename){
        dbx.filesUpload({
            path: "/" + filename,
            contents: content 
        })
    };
    ...

    ...

    Hi fangzefunny,

    In your 'saveDropbox' function, what are you doing with Dropbox 'filesUpload' method result? On success there should be received a file metadata to the file just uploaded and on error - the corresponding error information. As far as I can see, you are ignoring this information. In this context, you function always succeeds regardless if actual upload succeeds or fails. The information of the error pointing the reason of an upload fail is lost too. That's why you never get any error message and your 'success message' always appears. 😉 Be more careful.

    Hope this gives direction.


  • fangzefunny wrote:

    ...

    At the end of the experiment, the success message was always prompted. But the data was not always uploaded to my dropbox. 

    ...

    ...
    var saveDropbox = function(content, filename){
        dbx.filesUpload({
            path: "/" + filename,
            contents: content 
        })
    };
    ...

    ...

    Hi fangzefunny,

    In your 'saveDropbox' function, what are you doing with Dropbox 'filesUpload' method result? On success there should be received a file metadata to the file just uploaded and on error - the corresponding error information. As far as I can see, you are ignoring this information. In this context, you function always succeeds regardless if actual upload succeeds or fails. The information of the error pointing the reason of an upload fail is lost too. That's why you never get any error message and your 'success message' always appears. 😉 Be more careful.

    Hope this gives direction.

    • fangzefunny's avatar
      fangzefunny
      Explorer | Level 3

      Thanks! I found the reason today.
      Sometimes, people make too many mistakes, making the (adaptive) experiment too long, and that causes the data file being too large. The large file cannot pass through the "post" operation, causing the problem of 413.
      I was not able to see the error message before. I run it online, so I cannot see others' error messages. When I did it locally, I responded well, so the issue did not occur.

      Anyway, thanks! It is not a problem with Dropbox!

      • Здравко's avatar
        Здравко
        Legendary | Level 20

        fangzefunny wrote:

        ... The large file cannot pass through the "post" operation, causing the problem of 413. ...


        Hi fangzefunny,

        For large files consider using upload sessions instead, not single upload. Every upload transaction is guaranteed to pass for no more than 150MB. In a session you can split the file content on pieces 150MB each, at most. In such a way you can upload significantly larger file. 😉

        Good luck.