starred/ALwrity
1
0
Fork 0
mirror of https://github.com/AJaySi/ALwrity.git synced 2026-04-25 08:55:58 +03:00
Code Issues 82 Projects Releases 4 Packages Wiki Activity

[GH-ISSUE #123] Integrate scylla with proxy for web scraping #94

New issue
Closed
opened 2026-03-02 23:33:34 +03:00 by kerem · 1 comment
kerem commented 2026-03-02 23:33:34 +03:00
Owner
Copy link

Originally created by @AJaySi on GitHub (Sep 11, 2024).
Original GitHub issue: https://github.com/AJaySi/ALwrity/issues/123

Originally assigned to: @AJaySi on GitHub.

Scylla: A free, open-source proxy pool for scraping.

Originally created by @AJaySi on GitHub (Sep 11, 2024). Original GitHub issue: https://github.com/AJaySi/ALwrity/issues/123 Originally assigned to: @AJaySi on GitHub. [Scylla](https://github.com/imWildCat/scylla): A free, open-source proxy pool for scraping.
kerem closed this issue 2026-03-02 23:33:34 +03:00
kerem commented 2026-03-02 23:33:35 +03:00
Author
Owner
Copy link

@AJaySi commented on GitHub (Mar 13, 2025):

I did some research and ALwrity already has firecrawl, Exa and Tavily AI doing the same. There are free mostly and very generous in their free tier. Its better to use them then cook up our own. There are problems of getting banned etc with scraping, which ALwrity does not want concentrate on, right now.
As and when above, web researching API become paid/expensive, we will have to consider such libraries.

Closing for now.

<!-- gh-comment-id:2720041283 --> @AJaySi commented on GitHub (Mar 13, 2025): I did some research and ALwrity already has firecrawl, Exa and Tavily AI doing the same. There are free mostly and very generous in their free tier. Its better to use them then cook up our own. There are problems of getting banned etc with scraping, which ALwrity does not want concentrate on, right now. As and when above, web researching API become paid/expensive, we will have to consider such libraries. Closing for now.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
codex/task-title-3y5pbt
codex/task-title-lsrpho
codex/task-title
codex/locate-and-render-brollinfopanel-component
codex/import-scene-composition-symbols-in-broll_service
codex/refactor-preview-generation-flow
codex/standardize-chart-preview-route-usage
codex/import-b-roll-handler-and-update-router
codex/implement-podcast-research-response-updates
codex/add-cost-estimation-feature-to-podcastdashboard
codex/remove-heuristic-calculators-and-update-estimates-bhvhnh
codex/remove-heuristic-calculators-and-update-estimates
codex/implement-dedicated-podcastcostest-model
codex/add-podcast-maker-feature-documentation
codex/review-flat-file-context-system-implementation
codex/create-podcast-maker-best-practices-documentation
codex/add-podcast-maker-journey-pages-for-personas
codex/refactor-podcast-maker-documentation-structure
dependabot/npm_and_yarn/frontend/npm_and_yarn-7d3393088f
codex/log-skipped-and-loaded-items
codex/create-startup-phases-for-podcast-demo-tugrzv
codex/create-startup-phases-for-podcast-demo
codex/add-api-feature-profile-status-endpoint
codex/refactor-routermanager-to-use-registry
codex/add-new-modules-for-feature-management
codex/downgrade-api-key-warning-logging
codex/refactor-onboarding-loading-logic
codex/add-ci-lint-check-for-user_id-patterns
codex/remove-sessionlocal-import-and-add-health-check
codex/refactor-dubbed_audio_dir-to-tenant-path-resolver
codex/update-task-metadata-and-api-responses
codex/verify-podcast-analysis-and-cost-tracking
codex/assert-api-endpoints-on-startup
codex/add-env-controlled-guard-for-stripe-key
codex/remove-feature-flag-for-production-demo
codex/add-mode-aware-redirect-logic-in-pricingpage
codex/document-deployment-configuration-for-demo-mode
codex/ensure-api.subscription.router-always-included
codex/guard-onboarding-manager-and-endpoints
codex/wrap-router-inclusion-in-demo-mode-check
codex/add-podcast-only-demo-mode-toggle
dependabot/npm_and_yarn/frontend/npm_and_yarn-f5ea90b336
codex/update-reset_current_billing_period-logic
codex/update-error-handling-in-monitoring-middleware
codex/move-stripe_plan_price_mapping-to-env-driven-config
codex/refactor-billingservice-to-use-authenticated-user
codex/update-user-access-verification-for-routes
session/agent_eae6711f-e04b-4d48-8d66-43ee051feb7b
v0.5.0
v0.3
v0.2
v0.1
Labels
Clear labels   
AI Content Agents

   
AI Content Strategy

   
AI Content planning

   
AI Marketing Tools

   
AI SEO

   
AI personalization

   
AI writer

   
ALwrity Copi-lot

   
Alwrity web search

   
Anthropic

   
DeepSeek

   
Gemini AI

   
Integration

   
LLM

   
OnBoarding

   
OnBoarding

   
RAG knowledgebase Memory

   
bug

   
documentation

   
enhancement

   
good first issue

   
help wanted

   
invalid

   
openai

   
pull-request

Mirrored from GitHub Pull Request

No labels
AI Content Agents
AI Content Strategy
AI Content planning
AI Marketing Tools
AI SEO
AI personalization
AI writer
ALwrity Copi-lot
Alwrity web search
Anthropic
DeepSeek
Gemini AI
Integration
LLM
OnBoarding
OnBoarding
RAG knowledgebase Memory
bug
documentation
enhancement
good first issue
help wanted
invalid
openai
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/ALwrity#94
No description provided.