Refactor logging in utils and sohu_ccef modules; remove unused test.py file

Update sohu_ccef.py

add exponential backoff logic for persistent network issues

Add Data Source [CCEF]

Add Data Source [CCEF]

refactor logging and streamline content update process

update eastmoney.py

creation of flask api

Handle TimeoutError in crawl function to improve error handling

Fix retry logic in crawl function to decrement retries correctly

Increase timeout for URL requests in crawl functions to enhance reliability

Add timeout parameter to URL requests in crawl functions for improved reliability

Implement retry mechanism for URL requests in crawl function to enhance reliability

Add error handling for URL requests in crawl function

Handle IncompleteRead exception in crawl function for improved error handling

Refactor vectorizer.py by removing commented-out code and improve error handling in stats.py during URL requests

Remove openai_vectorize calls from crawl function and retain vectorize for article processing

Refactor vectorization process by removing openai_vectorize calls and updating vectorizer initialization

Refactor content update process to ensure reference ID is set to None and re-enable vectorization functions in article processing

Handle missing reference ID by setting it to None in article data and refactor URL construction for clarity

Add reference ID extraction and implement retry logic for document addition

Update article classification from 'site' to 'category' for policy interpretation

Add logging for DataFrame output in vectorize function and log content in _crawl function for better traceability

Refactor logging in multiple files to replace print statements with logging calls for better traceability

Handle exceptions in _crawl function by logging errors and returning None to prevent crashes

Implement retry mechanism in _crawl function to handle IncompleteRead exceptions and improve URL fetching reliability

Handle IncompleteRead exception in _crawl function to prevent crashes during URL fetching

Refactor exception handling in multiple files to specify exception types and improve logging

Refactor error handling and improve logging in utils.py; update vectorization process in vectorizer.py; adjust variable naming in eastmoney.py

Update eastmoney.py

Update csrc.py

add more logging statements

commit

fix logging issue

Update csrc.py

update flow

commit

commit

commit

commit