Skip to content

Double quote in string field #11

Open

Description

@JulesGrillot

When starting the query plpyodk.odk_central_to_pg, I got an error because of field value with double quotes :

ERROR: Token "berger" is invalid.syntaxe en entrée invalide pour le type json

ERREUR: syntaxe en entrée invalide pour le type json
SQL state: 22P02
Detail: Token "berger" is invalid.
Context: données JSON, ligne 1 : ...nger le brame du cerf. Recommande de lire "berger...
instruction SQL « DROP TABLE IF EXISTS odk_central.form;
CREATE TABLE IF NOT EXISTS odk_central.form AS
SELECT key as tablename, (json_array_elements(value)) as json_data
FROM json_each(plpyodk.get_complete_submissions_with_filter('project_id'::text, 'form'::text,''::text)::json)
»
fonction PL/pgSQL plpyodk.odk_central_to_pg(integer,text,text,text,text), ligne 3 à EXECUTE

Versions :
Postgresql 17.6
Ubuntu 22.04
pyodk v1.2.1

Activity

  1. JulesGrillot commented on Aug 26, 2025

    @JulesGrillot
    ContributorAuthor

    This is the modifications I made to get_complete_submissions_with_filter.sql to replace the double quotes :

    I added a function at the beginning of the file :

    import collections.abc
    
    def update(d):
        for k, v in d.items():
            if isinstance(v, collections.abc.Mapping):
                d[k] = update(v)
            else:
                if isinstance(v, str):
                    d[k] = v.replace('"', '')
                else:
                    d[k] = v
        return d

    And change this line with this code :

    value = [update(elem) for elem in response.json()['value']]

    Full code :

    import collections.abc
    
    def update(d):
        for k, v in d.items():
            if isinstance(v, collections.abc.Mapping):
                d[k] = update(v)
            else:
                if isinstance(v, str):
                    d[k] = v.replace('"', '')
                else:
                    d[k] = v
        return d
    
    
    def fresh_data_only(pid, fid, path, filter, datas):
        from pyodk.client import Client
        import re
        import json 
        client = Client()
        client.open()
        if path == '':
            return None
    
        if not filter:
            url = 'projects/'+pid+'/forms/'+fid+'.svc/'+path
        else:
            url = 'projects/'+pid+'/forms/'+fid+'.svc/'+path+'?$filter='+filter   
        
        if re.match(r"Submissions\?.*", path) or re.match(r".*\)$", path):
            tablename = 'submissions'
        else:
            tablename = path.rsplit('/')[-1]    
        
        response = client.get(url)
        response.raise_for_status()
    
        value = [update(elem) for elem in response.json()['value']]
    
        navigationlinks = re.findall(r'\'\w+@odata\.navigationLink\':\s+"([^"}]*)', str(value))
        result = list(dict.fromkeys(navigationlinks))
        for (link) in navigationlinks:
            link = link.replace("'", "'").replace('"','')
            fresh_data_only(project_id, form_id, link, '', datas)
        
        if tablename in datas.keys():
            datas[tablename] += value
        else:
            datas[tablename]=value
    		
        json_datas = str(json.dumps(datas))
        return json_datas.encode(encoding='utf-8').decode('unicode_escape')
    	
    return fresh_data_only(project_id, form_id, 'Submissions', filter, datas = {}).replace('\n', '\\n')

    I used this stackoverflow answer to help me make the function.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions